Optimization under Ordinal Scales: When is a Greedy Solution Optimal?

Transcription

1 Optimization under Ordinal Scales: When is a Greedy Solution Optimal? Aleksandar Pekeč BRICS Department of Computer Science University of Aarhus DK-8000 Aarhus C, Denmark pekec@brics.dk Abstract Mathematical formulation of an optimization problem often depends on data which can be measured in more than one acceptable way. If the conclusion of optimality depends on the choice of measure, then we should question whether it is meaningful to ask for an optimal solution. If a meaningful optimal solution exists and the objective function depends on data measured on an ordinal scale of measurement, then the greedy algorithm will give such a solution for a wide range of objective functions. Keywords: Optimization, Measurement Theory, Greedy Algorithms Basic Research in Computer Science, Centre of the Danish National Research Foundation 1

2 1 Introduction Mathematical formulation of an optimization problem often depends on data which can be measured in more than one acceptable way. For example, in the linear objective function P(x;w) = w T x = n i=1 w i x i, the weights w 1,..., w n often represent certain costs or profits related to the i-th variable. Obviously, we can express these costs (profits) in dollars or in cents or in any other currency. One should expect that the optimal solution will remain optimal regardless of which currency is used to represent the weights w 1,..., w n. More generally, the weights can measure preferences or priorities (weights are measured on an ordinal scale ). We will show that for a wide range of objective functions that depend on data measured on an ordinal scale, an optimal solution is invariant of the choice of an acceptable way to represent data if and only if it is obtained by the greedy algorithm. This result yields to a new characterization of optimisation problems for which the greedy algorithm produces an optimal solution. These questions are well studied in the case of the linear objective functions (for example, see Zimmermann (1977)) but there are almost no results of this type when the objective is non-linear (resource allocation problems with weakly concave objective functions for which the greedy algorithm gives an optimal solution are characterized in Fedregruen and Groenevelt (1986)). In the rest of this section we briefly introduce the language of measurement theory. There exists an extensive literature on this subject (e.g. Krantz et al. (1971), Suppes et al. (1989), Luce et al. (1990), Narens (1985), Pfanzagl (1968), Roberts (1979)), and we will follow Roberts (1979). A scale of measurement is a mapping which assigns (positive) real numbers to objects being measured in such a way that certain empirical relations are preserved. An admissible transformation of scale is a transformation f of scale values which leads to another acceptable mapping, i.e., one which again preserves the empirical relations. For example, all exchange rates are of the form f(x) = αx, α > 0. Similarly, we can measure mass in pounds or kilograms and here we have p(x) = 2.2x where x is the amount in kg. and p(x) is the amount in lb. The set of all admissible transformations of a particular scale of measurement defines a scale type. For example, if the set of all admissible transformations of the scale of measurement is {f(x) = αx : α > 0} we are talking about a ratio scale ( objects are measured on a ratio scale ). Measurement of mass, currency amounts, time (seconds, minutes, hours) are examples of ratio scales. An interval scale is a scale where the set of admissible transformations is {f(x) = αx + β : α > 0, β R}. Temperature is measured on an interval scale: for example, f(x) = 9 x + 32 is a transformation from 5 o F to o C. If we want to make sure that our optimization problem doesn t change an 2

3 optimal solution if we use an exchange rate and pay a flat fee for each transaction, then this is the same as to say that weights can be measured on an interval scale. In many instances very precise data is not available and the most that is given is a relationship among data. For example, we can only have ordering or a preference list of all the weights (priorities): w 1 w 2... w n (where we know if the inequality is strict or we have an equality) and any set of n numbers satisfying this relationship is an acceptable way to measure data. In such a case any increasing function f is an admissible transformation. We say that data is measured on an ordinal scale if {f : f is increasing} is the set of the admissible transformations. If we can conclude from the nature of the problem (or data available) that data are measured on some particular scale of measurement, then we should expect that an optimal solution remains optimal if we apply an admissible transformation of the scale to all of the data. In other words, we expect a solution to be optimal regardless of the choice of an acceptable way to measure data. For example, it would be meaningless to conclude that a solution is optimal if we measure our cost in dollars but it is not optimal if we measure it in cents. In measurement theory, we call a statement using scales meaningful if its truth or falsity is unchanged after applying admissible transformations to all of the scales in the statement. The meaningfulness of the conclusion of optimality for combinatorial optimization problems was studied by Roberts (1990,1994) and Pekeč (1996). In this paper we analyze the problems where data is assigned nonnegative real numbers and is measured on an ordinal scale. After formulating the problem precisely in Section 2, we turn to the question of optimization in Section 3. We analyze optimization under an ordinal scale of the problem formulated in Section 2 and show that only greedy solutions can be meaningful optimal solutions. This gives far reaching conclusions about the optimization (since there is an efficient algorithm for finding an optimal solution) just from knowing that some data is measured on an ordinal scale of measurement. 2 Formulation of the Problem Throughout x will denote a vector (x 1,..., x n ) T, and R n + will denote a set of all n-dimensional vectors with nonnegative coordinates. We will also use the notation f(w) := (f(w 1 ),..., f(w n )) where w R n and f : R R. 3

4 We consider an objective function P of the form P : X R n + R where X = X 1 X 2... X n and each X i is a metric space with a metric d i. We will analyze the optimization problem minp(x;w) (1) x F where w R n + is fixed and F X is the set of feasible solutions. Let us suppose that w 1, w 2,..., w n are measured on some scale of measurement. Then we say that x F is a meaningful optimal solution for problem (1) if x is an optimal solution for problem (1) and the conclusion of optimality is a meaningful statement. More formally, x F is a meaningful optimal solution for problem (1) if and only if P(x;w) P(z;w) P(x; Φ(w)) P(z; Φ(w)) for every z F, and every admissible transformation Φ of the scale of measurement of w 1, w 2,..., w n. Throughout we will assume that P has the following property: For every i there exists a unique a i X i such that x X, w R n + : P(x 1,..., x i 1, a i, x i+1,..., x n ;w) P(x;w) (2) We call a i the desirable value for the i-th variable. Let us make some additional reasonable assumptions about the function P. We will say that x = i y if x,y X and x j = y j for all j i. First we assume that P is increasing as the i-th variable is moving away from the desirable value a i. In other words, for every i: x,y X,x= i y, w R n + : P(x;w) P(y;w) d i (x i, a i ) d i (y i, a i ). (3) Note that this condition is equivalent to saying that there exists a function p:r 2n + R, nondecreasing in each of the first n coordinates, and such that P(x;w) = p(d 1 (x 1, a 1 ),..., d n (x n, a n );w). Obviously, if a = (a 1,..., a n ) F then the minimum is attained at a and we may assume that P(a;w) = 0 (otherwise we just define P (x;w) := P(x;w) P(a;w)) and, therefore, P 0. 4

5 We say that P(x;w) is a regular objective function if there are unique a 1 X 1, a 2 X 2,..., a n X n such that (2), and (3) hold for every i = 1, 2,..., n. The whole setup might look artificial but a wide range of combinatorial optimization problems can be formulated in this way. Example: Let X i = [a i, ) R for i = 1,..., n. It is easy to show that P(x;w) = w T x is a regular objective function. Therefore, the conditions (2), and (3) are satisfied for any linear programming problem or integer programming problem for which there exist a 1,..., a n such that F X. More generally, every summable, separable, increasing objective function, i.e., function of the form cj1,j 2,...,j k (w j1,..., w jk )f j1,j 2,...,j k (x j1,..., x jk ), (4) where c j1,j 2,...,j k : R k + R + and f j1,j 2,...,j k (x j1,..., x jk ) : X j1 X jk R + are increasing functions in every coordinate, is an example of a regular objective function (recall that X i = [a i, ) for i = 1,..., n). One of the simplest examples of this form is the objective function for quadratic optimization (P(x;w) = i,j w i w j x i x j ). 3 Optimization As mentioned in the previous section, if a = (a 1,..., a n ) F, then the problem is trivial since P achieves a global minimum at a. In general, any optimal solution will have the property that d i (x i, a i ) is as small as possible and variables with higher weights will tend to be closer to their desired value than variables with lower weights. One may try to attack this problem using a greedy heuristic: Greedy algorithm INPUT: Set F X, F closed in a metric space X. a i X i, i = 1,..., n. w i 0, i = 1,..., n. (a) Rename all the variables such that w 1 w 2... w n. (b) Let i = 1. Let F 1 = F 1. If F i, then find x i such that d i (x i, a i ) = min{d i (y i, a i ) : y F i }. (Note that the minimum exists since d i (, a i ) is a continuous function on X i and {y i : y F i } is closed in the metric space X i since F i is closed in X.) 5

6 2. If F i =, then STOP (Output: F =.) 3. If i = n then STOP (Output: x = (x 1,..., x n )) 4. Let F i+1 := {y F i : y i = x i }. (Note that F i+1 X is closed since both X 1 X i 1 {x i } X i+1 X n and F are closed in X.) 5. Let i := i + 1 and go to Step 1. If F = then the execution of the algorithm will STOP in Step 2 with i = 1 and the output will be F =. If F then, by induction on i, the set F i is nonempty for every i = 2,..., n and the output (x 1,... x n ) of the Greedy algorithm is feasible. The Greedy algorithm is not necessarily fast; in some cases determining x i might be an easy task but it might be as hard as the original problem if the set F i is complicated. Moreover, it is clear that there is no guarantee that the algorithm will ever produce an optimal solution. We say that x F is a greedy solution if it can be produced by the greedy algorithm. Note that a greedy solution might not be uniquely determined. Whenever w 1,..., w n are measured on an ordinal scale, something more can be said about greedy solution(s). However, we will need to impose new restrictions on P. We say that P(x;w) is w-unbounded if for every i = 1,..., n: w R n +, x,y X,x y, x = i y : lim P(x;w) P(y;w) = (5) w i Roughly speaking, P is w-unbounded if there can be an unbounded change in the value of P for any change of the value of the i-th variable. P(x;w) is controllable on X if moving a variable with small weight cannot result in big changes in the value of the objective function. More precisely, for every i = 1,..., n: x,y X,x= i y, M i (x,y), w R n + : lim P(x;w) P(y;w) < M i (6) w i 0 6

7 Example: It is straightforward to check that the linear objective function P(x; w) = w T x = w i x i and the quadratic objective function P(x,w) = w i w j x i x j are examples of the objective functions that are w-unbounded and controllable on R n +. Now we can state and prove the main result: Theorem 1 Consider optimization problem (1) where F is a closed set and P is regular, w-unbounded and controllable on X. If w 1,..., w n are measured on an ordinal scale then every meaningful optimal solution is a greedy solution. Proof: We will show that if z is not a greedy solution, then z is not a meaningful optimal solution. We first rename all the indices such that w 1 w 2... w n 0. (7) Note that this can be done uniquely if and only if all w i s are different. We say that the indexing is proper if (7) holds. Let z be a meaningful optimal solution. Let i be the smallest index such that for any proper indexing there is no greedy solution x with x k = z k for all k = 1,..., i. If there is no such i then z is a greedy solution. Note that i n since otherwise (3) (with i = n) and Step 1 of the algorithm would give P(x;w) P(z;w), so P(x;w) = P(z;w) for a greedy solution x. Moreover, z must be greedy since by (3), d n (x n, a n ) = d n (z n, a n ). Therefore, we may assume i < n. Let x be a greedy solution for which x k = z k for all k = 1,..., i 1. Now, for all w, P(z;w) P(x;w) = P(z;w) P(z 1,..., z n 1, x n ;w) +P(z 1,..., z n 1, x n ;w) P(z 1,..., z n 2, x n 1, x n ;w). +P(z 1,..., z i, x i+1,..., x n ;w) P(x;w). Since the w i are measured on an ordinal scale we can choose any w satisfying (7) and maintain the optimality of z. We choose w k, k = i +1,..., n, such that (7) holds and such that P(z 1,..., z k 1, z k, x k+1,..., x n ;w) P(z 1,..., z k 1, x k, x k+1,..., x n ;w) M k. We can do the latter because P is controllable on X. Now we have: P(z;w) P(x;w) M n... M i+2 M i+1 +P(z 1,..., z i, x i+1,..., x n ;w) P(x;w). 7

8 Since P is w-unbounded and since x k = z k for k = 1,..., i 1, we can choose w i such that P(z 1,..., z i, x i + 1,..., x n ;w) P(x;w) > M i+1 + M i M n and all other w k (k = 1,..., i 1) to satisfy (7). Then we have P(z;w) P(x;w) > 0 which is a contradiction of the optimality of z Before presenting a characterization of optimization problems for which every a solution is optimal if and only if it is greedy, we need a following Lemma. Lemma 2 Suppose that, for every k = 1,..., n and every c 0 the equation d k (y k, a k ) = c has at most one solution y k X k. Furthermore suppose that, for any x F, the function f x : R n + R defined by f x(w) = P(x;w) is continuous. If for any choice of w R n +, there exists a greedy solution to problem (1) which is an optimal solution, then, for any choice of w R n +, every greedy solution to problem (1) is an optimal solution. Proof: To prove the lemma it suffices to show that, for any choice of w R n +, P(x G ;w) = P(y G ;w) for any two greedy solutions x G and y G corresponding to w. Suppose that there exists w R n + and two greedy solutions x G and y G corresponding to w such that ǫ := P(y G ;w) P(x G ;w) > 0 (note that w 0). For this ǫ, there exist δ 1 > 0 and δ 2 > 0 such that w w 1 < δ 1 f xg (w ) f xg (w) < ǫ 2, w w 1 < δ 2 f yg (w ) f yg (w) < ǫ 2. Hence, w w 1 < min{δ 1, δ 2 } f yg (w ) > f xg (w ). (8) (This is because f yg (w ) > f yg (w) ǫ/2 = (f xg (w) + ǫ) ǫ/2 = f xg (w) + ǫ/2 > f xg (w ).) Without loss of generality we may assume that w 1 w 2... w n and that x G is the unique greedy solution corresponding to this ordering. (Given an ordering, there is a unique greedy solution.) Let δ 0 = w 1 if w 1 =... = w n, and otherwise let δ 0 := min{ w i w j : i, j [n], w i w j } > 0. 8

9 Now we choose any δ > 0 such that δ < min{δ 0, δ 1, δ 2 }. Such δ defines w(δ) whose coordinates are [w(δ)] i = w i + δ 2i, i = 1,..., n. Note that w(δ) > 0, w(δ) w 1 < δ, and [w(δ)] 1 > [w(δ)] 2 >... > [w(δ)] n. Note that x G is the unique greedy solution corresponding to w(δ) and, by hypothesis, we conclude that x G is an optimal solution to problem (1) with weights w(δ). In particular, we have f xg (w(δ)) f yg (w(δ)) and this contradicts (8). Corollary 3 Suppose that the conditions of both Theorem 1 and Lemma 2 hold. Then the following are equivalent: (a) For any choice of w 1,..., w n measured on an ordinal scale, any optimal solution to problem (1) is meaningful (b) For any choice of w 1,..., w n, any optimal solution to problem (1) is greedy. Proof: (a) (b) follows directly from Theorem 1. It remains to show (b) (a). Let x be an optimal solution for problem (1) with weights w R n +. Then, by (b), x = x G for some greedy solution x G. Note that, for any increasing function Φ such that Φ(w) R n +, x = x G is a greedy solution to problem (1) with weights Φ(w). By Lemma 2, we conclude that x = x G is an optimal solution to problem (1) with weights Φ(w). Hence, x is a meaningful optimal solution. In order to conclude that every meaningful optimal solution (if such a solution exists) is greedy, no information about the set of feasible solutions F is needed and the objective function P doesn t need to be linear. Therefore, we cannot hope for any kind of underlying structure on X just because w 1,..., w n are measured on an ordinal scale. However, for a specific optimization problem it might be much easier to prove x F is optimal x F is greedy, than to prove that every optimal solution is meaningful if w 1,..., w n are measured on an ordinal scale. For example, in the case of optimization of the linear objective function over the set F {0, 1} n, it is well known that x F is optimal x F is greedy if and only if F defines a matroid. Note that we cannot hope that Theorem 1 will be true under much weaker conditions on P. For example, we need w-unboundedness: Example: Let X = {0, 1} 3. Let F = {(1, 0, 0), (0, 1, 1), (1, 0, 1), (1, 1, 0), (1, 1, 1)}, P = c T x where c i = 2 1/(1+w i ), and let w 1 > w 2 > w 3 0. Note that P is regular and controllable on X but it is not w-unbounded. Then it is easy to check that the 9

10 greedy solution (0, 1, 1) is never optimal and the optimal one is (1, 0, 0). Note that this is a meaningful optimal solution as long as we measure weights on an ordinal scale. Unfortunately, a solution produced by the Greedy algorithm might not be unique. Rules for resolving situations where there are a few ways to continue execution of the algorithm may vary from problem to problem. In some cases ties among the w i can be resolved arbitrarily. An example of such a case is again the case of a 0-1 programming problem where F defines a matroid structure. Theorem 1 suggests a new approach which might be useful in some cases. If the conditions of Theorem 1 hold, the search for an optimal solution can be reduced to greedy solutions only. In many cases, only a slight modification of the Greedy algorithm will eliminate greedy solutions which are not optimal. Therefore, information that weights can be measured on an ordinal scale is valuable and can be the main step towards finding an efficient optimization algorithm. (For example, Mahadev, Pekeč and Roberts (1995) model and analyze a scheduling problem where it is not at all obvious that the Greedy algorithm gives an optimal solution but it can be proved that every optimal solution is meaningful if problem data are measured on an ordinal scale. They also show that, if there is a choice for x i in Step 1 of the Greedy algorithm, x i cannot be chosen arbitrarily.) Theorem 1 and Corollary 3 can be interpreted as negative results also. If we consider a problem where there is a choice of weights such that greedy algorithm doesn t give an optimal solution (almost every problem is like that, e.g. the traveling salesman problem) and the w i are measured on an ordinal scale, then we conclude that there cannot be a meaningful optimal solution. In other words, it doesn t make much sense to ask for an optimal solution. Therefore, for every non-trivial problem (non-trivial in the sense that the greedy solution is not necessarily an optimal solution) where w i are measured on an ordinal scale, the conclusion of optimality might be a meaningless conclusion. Acknowledgement The author gratefully acknowledges the support of the National Science Foundation under grant number SES to Rutgers University. He also thanks Dr. Fred S. Roberts and an anonymous referee for their helpful comments. 10

11 References [1] Federgruen, A., Groenevelt, H. (1986) The Greedy Procedure of Resource Allocation Problems: Necessary and Sufficient Conditions for Optimality. Oper. Res. 34: [2] Krantz D.H., Luce R.D., Suppes P., Tversky A. (1971) Foundations of Measurement, Vol. I. Academic Press, New York [3] Luce R.D., Krantz D.H., Suppes P., Tversky A. (1990) Foundations of Measurement, Vol. III. Academic Press, New York [4] Mahadev N.V.R., Pekeč A., Roberts F.S. (1995) On an Aircraft Scheduling Problem with Priorities and Earliness/Tardiness Penalties. Research Report, Rutgers Center for Operations Research RRR [5] Narens L. (1985) Abstract Measurement Theory. MIT Press, Cambridge, MA [6] Pfanzagl J. (1968), Theory of Measurement Wiley, New York [7] Pekeč A. (1996) Limitations on Conclusions from Combinatorial Optimization Models. Ph.D. thesis, Department of Mathematics, Rutgers University. [8] Roberts F.S. (1979) Measurement Theory. Addison-Wesley, Reading, MA [9] Roberts F.S. (1990) Meaningfulness of conclusions from combinatorial optimization. Discr. Appl. Math. 29: [10] Roberts F.S. (1994) Limitations of conclusions using scales of measurement. Handbooks in OR & MS, S.M. Pollock, M.H. Rothkopf, and A. Barnett,Eds. 6: [11] Suppes P., Krantz D.H., Luce R.D., Tversky A. (1989) Foundations of Measurement, Vol. II. Academic Press, New York [12] Zimmermann U. (1977) Some Partial Orders Related to Boolean Optimization and the Greedy Algorithm. Annals of Discr. Math. 1: