Operations Research Letters. A generalized Weiszfeld method for the multi-facility location problem

Size: px
Start display at page:

Download "Operations Research Letters. A generalized Weiszfeld method for the multi-facility location problem"

Transcription

1 Operations Research Letters 38 ) 7 4 Contents lists available at ScienceDirect Operations Research Letters journal homepage: A generalized Weiszfeld method for the multi-facility location problem Cem Iyigun a,, Adi Ben-Israel b a Department of Industrial Engineering, Middle East Technical University, Ankara, Turkey b RUTCOR Rutgers Center for Operations Research, Rutgers University, NJ, USA a r t i c l e i n f o a b s t r a c t Article history: Received 5 May 9 Accepted 5 October 9 Available online 6 November 9 Keywords: Fermat Weber location problem Multi-facility location problem Decomposition method Probabilistic assignments Weiszfeld method An iterative method is proposed for the K facilities location problem. The problem is relaxed using probabilistic assignments, depending on the distances to the facilities. The probabilities, that decompose the problem into K single-facility location problems, are updated at each iteration together with the facility locations. The proposed method is a natural generalization of the Weiszfeld method to several facilities. 9 Elsevier B.V. All rights reserved.. Introduction The Fermat Weber location problem also single-facility location problem) is to locate a facility that will serve optimally a set of customers, given by their locations and weights, in the sense of minimizing the weighted sum of distances traveled by the customers. A well known method for solving the problem is the Weiszfeld method [3], a gradient method that expresses and updates the sought center as a convex combination of the data points. The multi-facility location problem MFLP) is to locate a given) number of facilities to serve the customers as above. Each customer is assigned to a single facility, and the problem also called the location allocation problem) is to determine the optimal locations of the facilities, as well as the optimal assignments of customers assignment is absent in the single-facility case.) MFLP is NP hard, [6]. We relax it by replacing rigid assignments with probabilistic assignments, as in [4,3] and [8]. This allows a decomposition of MFLP into single-facility location problems, coupled by the membership probabilities that are updated at each iteration.. The problem Notation., K : {,,..., K}. x denotes the Euclidean norm of a vector x R n. The Euclidean distance dx, y) x y is used throughout. Corresponding author. addresses: iyigun@ie.metu.edu.tr C. Iyigun), adi.benisrael@gmail.com A. Ben-Israel) /$ see front matter 9 Elsevier B.V. All rights reserved. doi:.6/j.orl.9..5 Let X : {x i : i, N} be a set of N data points in R n, with given weights {w i > : i, N}. Typically, the points {x i } are the locations of customers, the weights {w i } are their demands. Given an integer K < N, the MFLP is to locate K facilities, and assign each customer to one facility, so as to minimize the sum of weighted distances min c,c,...,c K x i C k w i dx i, c k ) L.K ) where {c k } are the locations or centers) of the facilities, and C k is the cluster of customers that are assigned to the kth facility. For K, one gets the Fermat Weber location problem: given X and {w i : i, N} as above, find a point c R n minimizing the sum of weighted distances, min c R n w i dx i, c), L.) see [,3,4,3] and their references. If the points {x i } are not collinear, as is assumed throughout, the objective function of L.) f c) w i dx i, c) ) is strictly convex, and L.) has a unique optimal solution. The gradient of ) is undefined if c coincides with one of the data points {x i }. For c X, f c) w i x i c x i c, )

2 8 C. Iyigun, A. Ben-Israel / Operations Research Letters 38 ) 7 4 and the optimal center c, if not in X, is characterized by f c ), expressing it as a convex combination of the points x i, c λ i x i, with weights λ i w i / x i c that depend on c. w m / x m c m This circular result gives rise to the Weiszfeld iteration, [3], c + : Tc) 3) where c + is the updated center, c is the current center, and w i / x i c Tc) : x i, if c X; w m / x m c m c, if c X. In order to extend f c) to all c, Kuhn [] modified it as follows: f c) : Rc), where f c), if c X; Rc) : max {, R j R j w j } R j, if c x 5) j X, where R j : i j 4) w i x i x j x i x j ) 6) is the resultant force of N forces of magnitude w i and direction x i x j, i j. The following properties of the mappings R ), T ), the optimal center c and any point x j X were proved by Kuhn []: c c Rc). 7a) c conv X the convex hull of X). If c c then Tc) c. Conversely, if c X, Tc) c then c c. If Tc) c then f Tc)) < f c). 7b) 7c) 7d) x j c w j R j. 7e) If x j c, the direction of steepest descent of f at x j is R j / R j. If x j c there exists δ > such that 7f) < c x j T s c) x j > δ for some s. 7g) lim c x j Tc) x j c x j Rj w j. 7h) For any c, if no c r : T r c ) X, then lim c r c. 7i) r These results are generalized in Theorem to the case of several facilities. Remark. Another claim in [], that T r c ) c for all but a denumerable number of initial centers c, was refuted by Chandrasekaran and Tamir [6]. Convergence can be assured by modifying the algorithm 3) and 4) at a non-optimal center that coincides with a data point x j. Balas and Yu [] proposed moving from x j in the direction R j 6) of steepest descent, assuring a decrease of the objective, and non-return to x j by 7c) and 7d). Vardi and Zhang [9] guaranteed exit from x j by augmenting the objective function with a quadratic of the distances to the other data points. Convergence was also addressed by Ostresh [7], Eckhardt [3], Drezner [], Brimberg [5], Beck et al. [], and others. 3. Probabilistic assignments For < K < N, the problem L.K ) is NP hard, [6]. It can be solved polynomially in N for K, see [], and possibly for other given K. We relax the problem by using probabilistic or soft) assignments, with cluster membership probabilities, p k x) : Prob {x C k }, k, K, assumed to depend only on the distances {dx, c k ) : k, K} of the point x from the K centers. A reasonable assumption is assignment to a facility is more probable the closer it is and a simple way to model it, p k x) dx, c k ) Dx), k, K, 8) w where w is the weight of x, and D ) is a function of x, that does not depend on k. There are other ways to model assumption A), but 8) works well enough for our purposes. Model 8) expresses probabilities in terms of distances, displaying neutrality among facilities in the sense of the Choice Axiom of Luce, [5, Axiom ], see [9, Appendix A]. Other issues such as attractiveness, introduced in the Huff model [5,6], see also [9], are ignored. Using the fact that probabilities add to one, we get from 8), dx, c j ) p k x) /dx, c k) /dx, c l ) l j k l m l A), k, K, 9) dx, c m ) interpreted as p k x) if dx, c k ), i.e., x c k. In the special case K, dx, c ) p x) dx, c ) + dx, c ), dx, c ) p x) dx, c ) + dx, c ). From 8), we similarly get Dx) w K dx, c j ) j l m l ), ) dx, c m ) which is up to a constant) the harmonic mean of the distances {dx, c j ) : j, K}. In particular, Dx) w dx, c )dx, c ), for K. ) dx, c ) + dx, c ) The function ) is called the joint distance function JDF) at x. Abbreviating p k x) by p k, Eq. 8) is an optimality condition for the extremum problem { } min w p dx, k c k) : p k, p k, k, K 3) with variables {p k }. The squares of probabilities in 3) are explained as a device for smoothing the underlying objective, min { x c k : k, K}, see the seminal article by Teboulle [8].

3 C. Iyigun, A. Ben-Israel / Operations Research Letters 38 ) L.K ) can thus be approximated by the minimization problem min dx i, c k ) P.K ) s.t. p k x i ), i, N, p k x i ), k, K, i, N, with two sets of variables, the centers {c,..., c K } and probabilities {p k x i ) : k, K, i, N}, corresponding, respectively, to the centers and assignments of the original problem L.K ). Remark. Given a solution of P.K ), if each customer frequents the centers with the corresponding probabilities, then the expected sum of weighted distances is p k x i ) w i dx i, c k ), 4) which is no better than if each customer is assigned to the nearest facility, i.e. the one with the highest probability. Therefore, even if an optimal solution of P.K ) is also optimal for L.K ), the value 4) is just an upper bound on the optimal value of L.K ). Another way to see this is by rewriting 4), using 8) and ), p k x i ) w i dx i, c k ) K Dx i ) w i Dx i ) min {dx i, c k )}. 5) k,k The inequality follows since the harmonic mean is no less than the minimum. The larger K is, the better is the upper bound 4), because the harmonic mean is then closer to the minimum. 4. Probabilities and centers The objective function of P.K ) is denoted f c,..., c K ) : dx i, c k ). 6) A natural approach to solving P.K ), see e.g. [7], is to fix one set of variables, and minimize P.K ) with respect to the other set, then fix the other set, etc. We thus alternate between ) the probabilities problem, i.e. P.K ) with given centers, and ) the centers problem, P.K ) with given probabilities, and update their solutions as follows: Probabilities update. With the centers given, and the distances dx i, c k ) computed for all centers c k and data points x i, the minimizing probabilities are given explicitly by 9), dx i, c j ) p k x i ) j k l m l, k, K. 7) dx i, c m ) Centers update. Fixing the probabilities p k x i ) in P.K ), the objective function 6) is a separable function of the cluster centers, f c,..., c K ) : f k c k ), 8) where f k c) : x i c, k, K. 9) The centers problem thus decomposes into K problems of type L.), coupled by the probabilities. The gradient of 9), wherever it exists, is f k c) x i c x i c), k, K. ) Zeroing ), we get the optimal centers c k λ k x i ) x i, k, K, a) as convex combinations of the data points, with weights λ k x i ) given by λ k x i ) p k x i) w i / x i c k, k, K, i, N. b) p k x j) w j / x j c k j Eqs. a) and b) induce K mappings T k : c T k c), k, K, T k c) : p k x i) w i / x i c x i p k x j) w j / x j c j a) for c different than the data points {x j : j, N}, and by continuity, T k x j ) : x j, for all j, N. 5. Territories of facilities b) In the MFLP L.K ), each facility serves a cluster of customers, in a certain territory see the discussion by Huff [5]). These territories are given naturally by the membership probabilities of the approximate problem P.K ). For any two centers c j, c k the locus of the points x with p j x) p k x) is by 7) represented by the equation x c j x c k 3) and is thus a hyperplane line in R ). The portion of the hyperplane 3), where p j x) p k x) p m x), m j, k, 4) is either empty, bounded or unbounded. We call it the common boundary of the clusters C j and C k. The common boundaries constitute a Voronoi diagram of the centers. Each Voronoi cell is a polyhedron, and is the natural territory of its facility, see e.g. Figs. b) and b). 6. Contour approximation and uncertainty The JDF at a point was given in ). The JDF of the set X {x i : i, N} is defined as the sum, DX) : w i Dx i ). 5) It depends on the K centers, and has the following useful property: for optimal centers, most of the data points are contained in its lower level sets, see [3,8]. We call this property contour approximation. The JDF of the set X, 5), is thus a measure of the proximity of customers to their respective facilities: the lower the value of DX), the better is the set of centers {c k }. The JDF ) has the dimension of distance. Normalizing it, we get the dimensionless function

4 C. Iyigun, A. Ben-Israel / Operations Research Letters 38 ) a) Data points b) Probability level sets. Fig.. Illustration of Example a) Data points b) Probability level sets. Fig.. Illustration of Example. Ex) / /K K w K Dx) dx, c j )), 6) j with / interpreted as zero. Ex) is the harmonic mean of the distances {dx, c j ) : j, K} divided by their geometric mean, and can be written, using 9), as the geometric mean of the probabilities up to a constant), /K K Ex) K p j x)). 7) j It follows that Ex), with Ex) if any dx, c j ), i.e. if x is a cluster center, and Ex) if and only if the probabilities p j x) are all equal. In particular, for K, dx, c )dx, c ) Ex) dx, c ) + dx, c ) p x)p x). 8) Ex) represents the uncertainty of classifying the point x, indeed it can be written as { Ex) exp I px), K )}, 9) where I px), ) is the Kullback Leibler distance, [], between the distributions K px) p x), p x),..., p K x)) and K K, K,..., ). K The CUF of the data set X {x i : i, N} is defined as EX) : N Ex i ), 3) and is a monotone decreasing function of K, the number of clusters, decreasing from EX) for K ), to EX) for K N, the trivial case where every data point is its own cluster). The right number K of facilities to serve the given customers if not given) is in general determined by economic considerations operating costs, etc.) An intrinsic criterion for determining the optimal K is provided by the rate of decrease of EX), see e.g. Fig. 3d). 7. Optimality conditions and convergence The gradient ) is undefined /) if c coincides with any of the data points. We modify the gradient, following Kuhn [] [], and denote the modified gradient by R k. If a center c k is not one of the data points, we copy ) with a change of sign, R k c k ) : x i c k x i c k ). 3) Otherwise, if a center c k coincides with a data point x j then x j belongs with certainty to the kth cluster and, by 7), p k x j ), p m x j ) for all m k. 3) In this case, define The latter distribution,, is of maximal uncertainty. Therefore, K Ex) px). We call Ex) the classification K uncertainty function, abbreviated CUF, at x. R k x j ) : max { R j k w j, } Rj k R j k, where 33a)

5 C. Iyigun, A. Ben-Israel / Operations Research Letters 38 ) a) Data points b) Level sets of the JDF c) Level sets of the CUF. classification uncertanity number of clusters d) Validation. Fig. 3. Illustration of Example 3. R j k i j x i x j x i x j ). 33b) Therefore, if R j k < w j then R k x j ) ; otherwise, R k x j ) is a vector with magnitude R j k w j and direction R j k. For K, 33a) and 33b) reduce to their single-facility counterparts 5) and 6), respectively. The results 7a) 7i) are reproduced, for several facilities, in parts a) i) of the following theorem. Theorem. a) Given the data {x i : i, N} and {w i : i, N}, let {c k : k, K} be arbitrary points, and let the corresponding KN probabilities {p k x i )} be given by 7). Then, the condition R k c k ), for all k, K, 34) is necessary and sufficient for the points {c,..., c K } to minimize f c,..., c K ) of 8). b) The optimal centers {c k : k, K} are in the convex hull of the data points {x,..., x N }. c) If a center c is optimal, then it is a fixed point of T k of a)), T k c) c, for some k, K. 35) Conversely, if c {x j : j, N} and satisfies 35), then c is optimal. d) If T k c) c, then f k T k c)) < f k c), k, K. 36) e) If the data point x j is an optimal center, then w j R j. f) If the data point x j is not an optimal center, the direction of steepest descent of f at x j is Rj R j. g) Let x i be a data point that is not an optimal center c k, for a given k, K. Then, there exist δ > and an integer s > such that h) < x i c δ implies x i T s k c) δ and x i T s k c) > δ x j T k c) lim Rj k c x j x j c p k x for k, K, j, N. j) w j i) Given any c k, k, K, define the sequence {cr k T r k c) : k r,,...}. If no c r k is a data point, then lim r c r k c k, for some optimal centers {c,..., c K }. 8. A generalized Weiszfeld method for the multi-facility location problem The above results are implemented in an algorithm for solving P.K ). Algorithm. A generalized Weiszfeld method for several facilities Data: D {x i : i,n} data points locations of customers), {w i : i,n} weights, K the number of facilities ɛ > stopping criterion) Initialization: K arbitrary centers {c k : k,k}, Iteration: Step compute distances {dx, c k ) : k,k} for all x D Step compute probabilities {p k x) : x D, k,k} using 7)) Step 3 update the centers {c + : k T kc k ) : k,k}

6 C. Iyigun, A. Ben-Israel / Operations Research Letters 38 ) 7 4 Step 4 using a) and b)) if dc +, k c k) < ɛ stop return to step Writing a) as T k c) j j we get, from 3), x i c p k x j) w j x j c x i c p k x j) w j x j c x i c + c x i c) + c T k c) c + h k c) R k c) 37) with h k c) j p k x j) w j x j c 38) showing that Algorithm is a gradient method, following the direction of the resultant R k c) with step of length h k c) R k c), except for the data points {x j : j, N} which are left fixed by T k. Remark 3. a) Convergence can be established as in the Weiszfeld algorithm, by forcing a move away from non-optimal data points where the algorithm gets stuck by b)), using ideas of Balas and Yu [], or Vardi and Zhang [9], see Remark. b) The probabilities {p k x i ) : k, K, i, N} are not needed explicitly throughout the iterations. These probabilities are given by the distances {dx i, c k )}, see 7), and therefore the centers update in Step 3 requires only the distances. Step may thus be omitted. c) The final probabilities following the stop in Step 4) are needed for assigning customers to facilities. In the absence of capacity constraints, each customer is assigned to the nearest center, which by 8) has the highest probability. d) The algorithm is robust, since the weights in b) are inversely proportional to the distance, and as a result outliers are discounted, having little influence on the facility locations. e) Algorithm can be modified to account for capacity constraints on the facilities, as in [7]. This is necessary in capacitated MFLP where such constraints may force the customers to split their demands among several facilities, and travel farther than in the unconstrained case. Capacitated MFLP is handled in a sequel article. f) Algorithm can handle other models for assumption A), for example, 8) replaced by, p k x) φdx, c k )) Dx), k, K, w where φ ) is an increasing function, such as d α, α > or e d, see [3, Sections.8 and 3]. 9. Numerical examples The optimal solution has value 43.9, and the value of the worst stable solution is Algorithm was tried times on this P.3), using random initial centers, and a tolerance ɛ.. The optimal solution was found in 6% of the trials, the worst solution in.7% of the trials. The average number of iterations was 3. The optimal centers are shown in Fig. b), together with some level sets of the membership probability for each cluster. The three half-lines are the common boundaries of the clusters, a Voronoi diagram of the centers, partitioning the plane into territories served by each facility. Example. This example uses data from Eilon et al. [4, p. 83]. It is required to locate 5 facilities to serve the 5 customers shown in Fig. a). For the original problem L.5), different solutions were reported in [4, Table 4.], with an optimal value of 7.54, and the objective value at the worst solution is.99. We tested Algorithm on these data, times, with random initial centers and ɛ.. The average number of iterations was. An optimal solution with value of 7.5, slightly lower than the one reported in [4]) was found in % of the, trials. In.35% of the cases, the algorithm found the worst solution. Fig. b) shows the optimal centers found by Algorithm ), some probability level sets for each cluster, and the common boundaries of the centers. The Voronoi diagram has 4 segments and 4 half-lines common boundaries are empty.) Example 3. Fig. 3a) shows a simulated data of 45 customers, organized in 3 equal clusters. Problem P.3) was solved. Fig. 3b) shows the centers, and some level sets of the JDF ), illustrating the contour approximation property, namely that the lower level sets of the JDF capture the data points. Fig. 3c) shows some level sets of the CUF 7), with darker shades indicating higher uncertainty. Uncertainty is minimal Ex) ) if x is one of the centers. Note the patch of maximal uncertainty Ex).99) in the middle, where one is indifferent between the three centers. The right number of clusters known a priori in this simulated example) is determined by solving P.K ) for values of K,,..., calculating the CUF EX) of the whole dataset 3), and stopping when the marginal decrease of uncertainty is negligible. The results are plotted in Fig. 3d), confirming K 3 as correct. If the data are amorphous with no clear number of clusters, e.g. the data shown in Fig. a), then the graph of EX) does not give a clue. Acknowledgements Helpful comments and suggestions by Jack Brimberg, Zvi Drezner, Arie Tamir, Marc Teboulle and a referee are gratefully acknowledged. Appendix. Proof of Theorem Proof of part a). If c k is not one of the data points, then R k c k ) is the gradient ) at c k, and 34) is both necessary and sufficient for a minimum, by the convexity of f k. If c k coincides with a data point x j, consider the change from x j to x j + t z where z. Then, d dt f kx j + t z) p k x j) w j R j k z. 39) t The greatest decrease of f k is for z along R j k, i.e., when Example. This example uses the data of Cooper [7]. It is required to locate 3 facilities to serve the 5 customers shown in Fig. a). z Rj k R j k, proving part f)).

7 C. Iyigun, A. Ben-Israel / Operations Research Letters 38 ) Therefore, c k that coincides with x j ) is a local minimum if and only if, p k x j) w j Rj k Rj k R j k, which is equivalent to R j k p k x j) w j, or R k c k ), by 33a). Proof of part b). If c k is one of the data points, then it is trivially in the convex hull. Otherwise, the condition R k c k ), see 34), results in a) and b) as above. Proof of part c). Follows from part a), since for c X, c T k c) R k c), while for c X, c T k c) for all k. Proof of part d). The two functions f k ) in 36) are different because the probabilities changed. We prove the inequality 36) for the original probabilities, noting that the updated probabilities 7) are optimal, and result in further decrease. c is not one of the data points, since T k c) c. It follows then from a) that T k c) is the center of gravity of weights / x i c placed at the data points x i. By elementary calculus, T k c) is the unique minimum of the strictly convex function gy) Since c T k c), gt k c)) x i c x i y. On the other hand, x i c x i T k c) < gc) x i c x i c f k c). p k gt k c)) x i) w [ )] i x i c + x i T k c) x i c x i c ) f k c) + f k T k c)) f k c) + Combining these results, f k T k c)) + proving that f k T k c)) < f k c). p k x i) w [ ] i x i T k c) x i c x i c p k x i) w [ i x i T k c) x i c ] < fk c) x i c Proof of part e). Follows from part a), the definition R k x j ) : max { R j k p k x j) w j, } Rj k R j k, and p k x j ), since x j c k. Proof of part f). This was shown in part a). 33a) Proof of part g). T k c) x i c + h k c) R k c) x i p k h k c) x j) w j x j c x j c) + j i hk c) x i c Since x i is not optimal, we have p k x j) w j x j x i x j x i ) > p k x i) w i. j i ) x i c). Hence, there exist δ > and ɛ > such that p k x j) w j x j c x j c) + ɛ) p k x i) w i j i for x i c δ. By the definition of h k, we have h k c)p k lim x i) w i. c x i x i c Hence, there exists δ > such that h k c) x i c < ɛ for < x i c δ. + ɛ) Set δ minδ, δ ). For < x i c δ, we have x i T k c) > h k c) + ɛ)p k x ɛ i) w i + ɛ) x i c ) ɛ > + ɛ) x i c + ɛ) ɛ + ɛ) x i c + ɛ) x i c. Since x i c >, + ɛ) t x i c > δ for some positive integer t and hence x i T s k c) > δ for some positive integer s with x i T s k c) δ. Proof of part h). For c not a data point, p k T k c) x / ) i) w i N p k x m) w m x i c x m c i j x i c m x i x j ) + x j m T k c) x j p k x m) w m x m c i j T kc) x j x j c i j w j + x j c p k x j) w j x i c x i p k x / i) w i N x i c x p k i x j ) x m) w m x m m c, / x i c x i x j ) i j p k x ) i) w i. x i c Taking the limits of the lengths of both sides, lim c x j x j T k c) x j c Rj k p k x j) w j. p k x j)

8 4 C. Iyigun, A. Ben-Israel / Operations Research Letters 38 ) 7 4 Proof of part i). With the possible exception of c k, the sequence {c r k } lies in the convex hull of vertices, a compact set. By the Bolzano Weierstrass theorem, there exits a convergent subsequence {c l : l,,.. k.}, such that lim l c l k c k. We prove that c k c k. If c l+ k T k c l) k cl k for some l, then the sequence repeats from that point and c k c l k. Since cl k is not a data point, c k c k by Theorem c). Otherwise, by Theorem d), f k c ) > k f kc ) > > k f kc r ) > > k f kc k ). 4) Hence lim fk c l ) k f kt k c l ))) k. 4) l The continuity of T k implies lim T kc l ) k T kc ) k 4) l we have f k c ) k f kt k c k )). 43) x j c l k Therefore, by Theorem d), c k T k c k ). If c k is not a data point, then c k c k by Theorem c). In any event, c k lies in the finite set of points {x,..., x N, c k }, where c k may be a data point. The only case that remains is c k x j for some j. If x j c k, we first isolate x j from the other data points and c k if it is not a data point) by a δ-neighborhood that satisfies Theorem g). Then, it is clear that we can choose our subsequence c l k x j such that x j T k c l) > δ k for all l. This means that the ratio x j T k c l k ) is unbounded. However, this contradicts Theorem h). Hence, x j c k and the theorem is proved. References [] E. Balas, C.-S. Yu, A note on the Weiszfeld-Kuhn algorithm for the general Fermat problem, Tech. Report, Carnegie Melon University, September 98. [] A. Beck, M. Teboulle, Z. Chikishev, Iterative minimization schemes for solving the single source localization problem, SIAM J. Optim. 9 8) [3] A. Ben-Israel, C. Iyigun, Probabilistic distance clustering, J. Classification 5 8) 5 6. [4] J.C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum, 98. [5] J. Brimberg, Further notes on convergence of the Weiszfeld algorithm, Yugoslav J. Oper. Res. 3 3) [6] R. Chandrasekaran, A. Tamir, Open questions concerning Weiszfeld s algorithm for the Fermat Weber location problem, Math. Program ) [7] L. Cooper, Heuristic methods for location allocation problems, SIAM Rev ) [8] T. Drezner, Z. Drezner, The gravity p median model, European J. Oper. Res. 79 7) [9] T. Drezner, Z. Drezner, S. Salhi, Solving the multiple competitive facilities location problem, European J. Oper. Res. 4 ) [] Z. Drezner, The planar two-center and two-median problems, Transport. Sci ) [] Z. Drezner, A note on the Weber location problem, Ann. Oper. Res. 4 99) [] Z. Drezner, K. Klamroth, A. Schöbel, G.O. Wesolowsky, The Weber problem, in: Z. Drezner, H.W. Hamacher Eds.), Facility Location: Applications and Theory, Springer,. [3] I.N. Eckhardt, Weber s problem and Weiszfeld s algorithm in general spaces, Math. Program. 8 98) [4] S. Eilon, C.D.T. Watson Gandy, N. Christofides, Distribution Management: Mathematical Modelling and Practical Analysis, Hafner, 97. [5] D.L. Huff, Defining and estimating a trading area, J. Marketing 8 964) [6] D.L. Huff, A programmed solution for approximating an optimum retail location, Land Econom ) [7] C. Iyigun, A. Ben-Israel, Probabilistic distance clustering adjusted for cluster size, Probab. Engrg. Info. Sci. 8) 9. [8] C. Iyigun, A. Ben-Israel, Contour approximation of data: a duality theory, Lin. Algeb. Appl. 43 9) [9] C. Iyigun, A. Ben-Israel, Probabilistic semi-supervised clustering and the uncertainty of classification, in: A. Fink, B. Lausen, W. Seidel, A. Ultsch Eds.), Advances in Data Analysis, Data Handling and Business Intelligence, Springer, 9. [] H.W. Kuhn, On a pair of dual nonlinear programs, in: J. Abadie Ed.), Methods of Nonlinear Programming, North-Holland, 967, pp [] H.W. Kuhn, A note on Fermat s problem, Math. Program ) [] S. Kullback, R.A. Leibler, On information and sufficiency, Ann. Math. Statist. 95) [3] Y. Levin, A. Ben-Israel, A heuristic method for large-scale multi-facility location problems, Comput. Oper. Res. 3 4) [4] R.F. Love, J.G. Morris, G.O. Wesolowsky, Facilities Location: Models and Methods, North-Holland, 988. [5] R.D. Luce, Individual Choice Behavior, Wiley, 959. [6] N. Megiddo, K.J. Supowit, On the complexity of some common geometric location problems, SIAM J. Comput ) [7] L.M. Ostresh, On the convergence of a class of iterative methods for solving the Weber location problem, Oper. Res ) [8] M. Teboulle, A unified continuous optimization framework for center-based clustering methods, J. Machine Learning 8 7) 65. [9] Y. Vardi, C.-H. Zhang, A modified Weiszfeld algorithm for the Fermat Weber location problem, Math. Program. A9 ) [3] E. Weiszfeld, Sur le point par lequel la somme des distances de n points donnés est minimum, Tohoku Math. J ) [3] G.O. Wesolowsky, The Weber problem: Its history and perspectives, Location Sci. 993) 5 3.

The Newton Bracketing method for the minimization of convex functions subject to affine constraints

The Newton Bracketing method for the minimization of convex functions subject to affine constraints Discrete Applied Mathematics 156 (2008) 1977 1987 www.elsevier.com/locate/dam The Newton Bracketing method for the minimization of convex functions subject to affine constraints Adi Ben-Israel a, Yuri

More information

THE NEWTON BRACKETING METHOD FOR THE MINIMIZATION OF CONVEX FUNCTIONS SUBJECT TO AFFINE CONSTRAINTS

THE NEWTON BRACKETING METHOD FOR THE MINIMIZATION OF CONVEX FUNCTIONS SUBJECT TO AFFINE CONSTRAINTS THE NEWTON BRACKETING METHOD FOR THE MINIMIZATION OF CONVEX FUNCTIONS SUBJECT TO AFFINE CONSTRAINTS ADI BEN-ISRAEL AND YURI LEVIN Abstract. The Newton Bracketing method [9] for the minimization of convex

More information

A modified Weiszfeld algorithm for the Fermat-Weber location problem

A modified Weiszfeld algorithm for the Fermat-Weber location problem Math. Program., Ser. A 9: 559 566 (21) Digital Object Identifier (DOI) 1.17/s1171222 Yehuda Vardi Cun-Hui Zhang A modified Weiszfeld algorithm for the Fermat-Weber location problem Received: September

More information

ALGORITHM FOR CONSTRAINED WEBER PROBLEM WITH FEASIBLE REGION BOUNDED BY ARCS. Lev A. Kazakovtsev. 1. Introduction

ALGORITHM FOR CONSTRAINED WEBER PROBLEM WITH FEASIBLE REGION BOUNDED BY ARCS. Lev A. Kazakovtsev. 1. Introduction FACTA UNIVERSITATIS (NIŠ) Ser. Math. Inform. Vol. 28, No 3 (2013), 271 284 ALGORITHM FOR CONSTRAINED WEBER PROBLEM WITH FEASIBLE REGION BOUNDED BY ARCS Lev A. Kazakovtsev Abstract. The author proposes

More information

Institut für Numerische und Angewandte Mathematik

Institut für Numerische und Angewandte Mathematik Institut für Numerische und Angewandte Mathematik When closest is not always the best: The distributed p-median problem Brimberg, J.; Schöbel, A. Nr. 1 Preprint-Serie des Instituts für Numerische und Angewandte

More information

minimize x subject to (x 2)(x 4) u,

minimize x subject to (x 2)(x 4) u, Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for

More information

ICS-E4030 Kernel Methods in Machine Learning

ICS-E4030 Kernel Methods in Machine Learning ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This

More information

On stars and Steiner stars

On stars and Steiner stars On stars and Steiner stars Adrian Dumitrescu Csaba D. Tóth Guangwu Xu March 9, 9 Abstract A Steiner star for a set P of n points in R d connects an arbitrary point in R d to all points of P, while a star

More information

Linear Discrimination Functions

Linear Discrimination Functions Laurea Magistrale in Informatica Nicola Fanizzi Dipartimento di Informatica Università degli Studi di Bari November 4, 2009 Outline Linear models Gradient descent Perceptron Minimum square error approach

More information

On stars and Steiner stars. II

On stars and Steiner stars. II On stars and Steiner stars. II Adrian Dumitrescu Csaba D. Tóth Guangwu Xu June 6, 8 Abstract A Steiner star for a set P of n points in R d connects an arbitrary center point to all points of P, while a

More information

YURI LEVIN, MIKHAIL NEDIAK, AND ADI BEN-ISRAEL

YURI LEVIN, MIKHAIL NEDIAK, AND ADI BEN-ISRAEL Journal of Comput. & Applied Mathematics 139(2001), 197 213 DIRECT APPROACH TO CALCULUS OF VARIATIONS VIA NEWTON-RAPHSON METHOD YURI LEVIN, MIKHAIL NEDIAK, AND ADI BEN-ISRAEL Abstract. Consider m functions

More information

Operations Research Letters

Operations Research Letters Operations Research Letters 37 (2009) 1 6 Contents lists available at ScienceDirect Operations Research Letters journal homepage: www.elsevier.com/locate/orl Duality in robust optimization: Primal worst

More information

CS-E4830 Kernel Methods in Machine Learning

CS-E4830 Kernel Methods in Machine Learning CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This

More information

On stars and Steiner stars. II

On stars and Steiner stars. II Downloaded //7 to 69 Redistribution subject to SIAM license or copyright; see http://wwwsiamorg/journals/ojsaphp Abstract On stars and Steiner stars II Adrian Dumitrescu Csaba D Tóth Guangwu Xu A Steiner

More information

Linear & nonlinear classifiers

Linear & nonlinear classifiers Linear & nonlinear classifiers Machine Learning Hamid Beigy Sharif University of Technology Fall 1396 Hamid Beigy (Sharif University of Technology) Linear & nonlinear classifiers Fall 1396 1 / 44 Table

More information

Statistical Machine Learning from Data

Statistical Machine Learning from Data Samy Bengio Statistical Machine Learning from Data 1 Statistical Machine Learning from Data Support Vector Machines Samy Bengio IDIAP Research Institute, Martigny, Switzerland, and Ecole Polytechnique

More information

A Unified Approach to Proximal Algorithms using Bregman Distance

A Unified Approach to Proximal Algorithms using Bregman Distance A Unified Approach to Proximal Algorithms using Bregman Distance Yi Zhou a,, Yingbin Liang a, Lixin Shen b a Department of Electrical Engineering and Computer Science, Syracuse University b Department

More information

Support Vector Machine (SVM) and Kernel Methods

Support Vector Machine (SVM) and Kernel Methods Support Vector Machine (SVM) and Kernel Methods CE-717: Machine Learning Sharif University of Technology Fall 2014 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-Margin

More information

Optimization Tutorial 1. Basic Gradient Descent

Optimization Tutorial 1. Basic Gradient Descent E0 270 Machine Learning Jan 16, 2015 Optimization Tutorial 1 Basic Gradient Descent Lecture by Harikrishna Narasimhan Note: This tutorial shall assume background in elementary calculus and linear algebra.

More information

Robust linear optimization under general norms

Robust linear optimization under general norms Operations Research Letters 3 (004) 50 56 Operations Research Letters www.elsevier.com/locate/dsw Robust linear optimization under general norms Dimitris Bertsimas a; ;, Dessislava Pachamanova b, Melvyn

More information

Easy and not so easy multifacility location problems... (In 20 minutes.)

Easy and not so easy multifacility location problems... (In 20 minutes.) Easy and not so easy multifacility location problems... (In 20 minutes.) MINLP 2014 Pittsburgh, June 2014 Justo Puerto Institute of Mathematics (IMUS) Universidad de Sevilla Outline 1 Introduction (In

More information

Support Vector Machine (SVM) and Kernel Methods

Support Vector Machine (SVM) and Kernel Methods Support Vector Machine (SVM) and Kernel Methods CE-717: Machine Learning Sharif University of Technology Fall 2015 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-Margin

More information

L5 Support Vector Classification

L5 Support Vector Classification L5 Support Vector Classification Support Vector Machine Problem definition Geometrical picture Optimization problem Optimization Problem Hard margin Convexity Dual problem Soft margin problem Alexander

More information

IMM Technical Report Locating a circle on a sphere

IMM Technical Report Locating a circle on a sphere IMM Technical Report 003-7 Locating a circle on a sphere Jack Brimberg Royal Military College of Canada Henrik Juel Technical University of Denmark Anita Schöbel University of Kaiserslautern Abstract We

More information

Convex Optimization. Newton s method. ENSAE: Optimisation 1/44

Convex Optimization. Newton s method. ENSAE: Optimisation 1/44 Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)

More information

Set, functions and Euclidean space. Seungjin Han

Set, functions and Euclidean space. Seungjin Han Set, functions and Euclidean space Seungjin Han September, 2018 1 Some Basics LOGIC A is necessary for B : If B holds, then A holds. B A A B is the contraposition of B A. A is sufficient for B: If A holds,

More information

Machine Learning. Support Vector Machines. Fabio Vandin November 20, 2017

Machine Learning. Support Vector Machines. Fabio Vandin November 20, 2017 Machine Learning Support Vector Machines Fabio Vandin November 20, 2017 1 Classification and Margin Consider a classification problem with two classes: instance set X = R d label set Y = { 1, 1}. Training

More information

A Combinatorial Algorithm for the 1-Median Problem in R d with the Chebyshev-Norm

A Combinatorial Algorithm for the 1-Median Problem in R d with the Chebyshev-Norm A Combinatorial Algorithm for the 1-Median Problem in R d with the Chebyshev-Norm Johannes Hatzl a,, Andreas Karrenbauer b a Department of Optimization Discrete Mathematics, Graz University of Technology,

More information

min f(x). (2.1) Objectives consisting of a smooth convex term plus a nonconvex regularization term;

min f(x). (2.1) Objectives consisting of a smooth convex term plus a nonconvex regularization term; Chapter 2 Gradient Methods The gradient method forms the foundation of all of the schemes studied in this book. We will provide several complementary perspectives on this algorithm that highlight the many

More information

Clustering K-means. Clustering images. Machine Learning CSE546 Carlos Guestrin University of Washington. November 4, 2014.

Clustering K-means. Clustering images. Machine Learning CSE546 Carlos Guestrin University of Washington. November 4, 2014. Clustering K-means Machine Learning CSE546 Carlos Guestrin University of Washington November 4, 2014 1 Clustering images Set of Images [Goldberger et al.] 2 1 K-means Randomly initialize k centers µ (0)

More information

Unsupervised Learning

Unsupervised Learning 2018 EE448, Big Data Mining, Lecture 7 Unsupervised Learning Weinan Zhang Shanghai Jiao Tong University http://wnzhang.net http://wnzhang.net/teaching/ee448/index.html ML Problem Setting First build and

More information

Principles of Pattern Recognition. C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata

Principles of Pattern Recognition. C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata Principles of Pattern Recognition C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata e-mail: murthy@isical.ac.in Pattern Recognition Measurement Space > Feature Space >Decision

More information

Kaisa Joki Adil M. Bagirov Napsu Karmitsa Marko M. Mäkelä. New Proximal Bundle Method for Nonsmooth DC Optimization

Kaisa Joki Adil M. Bagirov Napsu Karmitsa Marko M. Mäkelä. New Proximal Bundle Method for Nonsmooth DC Optimization Kaisa Joki Adil M. Bagirov Napsu Karmitsa Marko M. Mäkelä New Proximal Bundle Method for Nonsmooth DC Optimization TUCS Technical Report No 1130, February 2015 New Proximal Bundle Method for Nonsmooth

More information

A convergence result for an Outer Approximation Scheme

A convergence result for an Outer Approximation Scheme A convergence result for an Outer Approximation Scheme R. S. Burachik Engenharia de Sistemas e Computação, COPPE-UFRJ, CP 68511, Rio de Janeiro, RJ, CEP 21941-972, Brazil regi@cos.ufrj.br J. O. Lopes Departamento

More information

Linear & nonlinear classifiers

Linear & nonlinear classifiers Linear & nonlinear classifiers Machine Learning Hamid Beigy Sharif University of Technology Fall 1394 Hamid Beigy (Sharif University of Technology) Linear & nonlinear classifiers Fall 1394 1 / 34 Table

More information

The Fundamental Welfare Theorems

The Fundamental Welfare Theorems The Fundamental Welfare Theorems The so-called Fundamental Welfare Theorems of Economics tell us about the relation between market equilibrium and Pareto efficiency. The First Welfare Theorem: Every Walrasian

More information

THECENTRALWAREHOUSELOCATION PROBLEM REVISITED

THECENTRALWAREHOUSELOCATION PROBLEM REVISITED THECENTRALWAREHOUSELOCATION PROBLEM REVISITED ZVI DREZNER College of Business and Economics California State University-Fullerton Fullerton, CA 92834 Tel: (714) 278-2712, e-mail: zdrezner@fullerton.edu

More information

Clustering K-means. Machine Learning CSE546. Sham Kakade University of Washington. November 15, Review: PCA Start: unsupervised learning

Clustering K-means. Machine Learning CSE546. Sham Kakade University of Washington. November 15, Review: PCA Start: unsupervised learning Clustering K-means Machine Learning CSE546 Sham Kakade University of Washington November 15, 2016 1 Announcements: Project Milestones due date passed. HW3 due on Monday It ll be collaborative HW2 grades

More information

AN ALTERNATING MINIMIZATION ALGORITHM FOR NON-NEGATIVE MATRIX APPROXIMATION

AN ALTERNATING MINIMIZATION ALGORITHM FOR NON-NEGATIVE MATRIX APPROXIMATION AN ALTERNATING MINIMIZATION ALGORITHM FOR NON-NEGATIVE MATRIX APPROXIMATION JOEL A. TROPP Abstract. Matrix approximation problems with non-negativity constraints arise during the analysis of high-dimensional

More information

Machine Learning Lecture 5

Machine Learning Lecture 5 Machine Learning Lecture 5 Linear Discriminant Functions 26.10.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course Outline Fundamentals Bayes Decision Theory

More information

Algorithms for Nonsmooth Optimization

Algorithms for Nonsmooth Optimization Algorithms for Nonsmooth Optimization Frank E. Curtis, Lehigh University presented at Center for Optimization and Statistical Learning, Northwestern University 2 March 2018 Algorithms for Nonsmooth Optimization

More information

A CHARACTERIZATION OF STRICT LOCAL MINIMIZERS OF ORDER ONE FOR STATIC MINMAX PROBLEMS IN THE PARAMETRIC CONSTRAINT CASE

A CHARACTERIZATION OF STRICT LOCAL MINIMIZERS OF ORDER ONE FOR STATIC MINMAX PROBLEMS IN THE PARAMETRIC CONSTRAINT CASE Journal of Applied Analysis Vol. 6, No. 1 (2000), pp. 139 148 A CHARACTERIZATION OF STRICT LOCAL MINIMIZERS OF ORDER ONE FOR STATIC MINMAX PROBLEMS IN THE PARAMETRIC CONSTRAINT CASE A. W. A. TAHA Received

More information

Lecture Notes on Support Vector Machine

Lecture Notes on Support Vector Machine Lecture Notes on Support Vector Machine Feng Li fli@sdu.edu.cn Shandong University, China 1 Hyperplane and Margin In a n-dimensional space, a hyper plane is defined by ω T x + b = 0 (1) where ω R n is

More information

ECE521 week 3: 23/26 January 2017

ECE521 week 3: 23/26 January 2017 ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear

More information

Machine Learning. Support Vector Machines. Manfred Huber

Machine Learning. Support Vector Machines. Manfred Huber Machine Learning Support Vector Machines Manfred Huber 2015 1 Support Vector Machines Both logistic regression and linear discriminant analysis learn a linear discriminant function to separate the data

More information

Relaxations of linear programming problems with first order stochastic dominance constraints

Relaxations of linear programming problems with first order stochastic dominance constraints Operations Research Letters 34 (2006) 653 659 Operations Research Letters www.elsevier.com/locate/orl Relaxations of linear programming problems with first order stochastic dominance constraints Nilay

More information

A Tabu Search Approach based on Strategic Vibration for Competitive Facility Location Problems with Random Demands

A Tabu Search Approach based on Strategic Vibration for Competitive Facility Location Problems with Random Demands A Tabu Search Approach based on Strategic Vibration for Competitive Facility Location Problems with Random Demands Takeshi Uno, Hideki Katagiri, and Kosuke Kato Abstract This paper proposes a new location

More information

Numerical Optimization

Numerical Optimization Constrained Optimization Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Constrained Optimization Constrained Optimization Problem: min h j (x) 0,

More information

ML (cont.): SUPPORT VECTOR MACHINES

ML (cont.): SUPPORT VECTOR MACHINES ML (cont.): SUPPORT VECTOR MACHINES CS540 Bryan R Gibson University of Wisconsin-Madison Slides adapted from those used by Prof. Jerry Zhu, CS540-1 1 / 40 Support Vector Machines (SVMs) The No-Math Version

More information

Spectral gradient projection method for solving nonlinear monotone equations

Spectral gradient projection method for solving nonlinear monotone equations Journal of Computational and Applied Mathematics 196 (2006) 478 484 www.elsevier.com/locate/cam Spectral gradient projection method for solving nonlinear monotone equations Li Zhang, Weijun Zhou Department

More information

Machine Learning A Geometric Approach

Machine Learning A Geometric Approach Machine Learning A Geometric Approach CIML book Chap 7.7 Linear Classification: Support Vector Machines (SVM) Professor Liang Huang some slides from Alex Smola (CMU) Linear Separator Ham Spam From Perceptron

More information

SOME STABILITY RESULTS FOR THE SEMI-AFFINE VARIATIONAL INEQUALITY PROBLEM. 1. Introduction

SOME STABILITY RESULTS FOR THE SEMI-AFFINE VARIATIONAL INEQUALITY PROBLEM. 1. Introduction ACTA MATHEMATICA VIETNAMICA 271 Volume 29, Number 3, 2004, pp. 271-280 SOME STABILITY RESULTS FOR THE SEMI-AFFINE VARIATIONAL INEQUALITY PROBLEM NGUYEN NANG TAM Abstract. This paper establishes two theorems

More information

Nonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control

Nonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control Nonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control 19/4/2012 Lecture content Problem formulation and sample examples (ch 13.1) Theoretical background Graphical

More information

Model Complexity of Pseudo-independent Models

Model Complexity of Pseudo-independent Models Model Complexity of Pseudo-independent Models Jae-Hyuck Lee and Yang Xiang Department of Computing and Information Science University of Guelph, Guelph, Canada {jaehyuck, yxiang}@cis.uoguelph,ca Abstract

More information

A class of Smoothing Method for Linear Second-Order Cone Programming

A class of Smoothing Method for Linear Second-Order Cone Programming Columbia International Publishing Journal of Advanced Computing (13) 1: 9-4 doi:1776/jac1313 Research Article A class of Smoothing Method for Linear Second-Order Cone Programming Zhuqing Gui *, Zhibin

More information

Constrained Optimization and Lagrangian Duality

Constrained Optimization and Lagrangian Duality CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may

More information

Optimization methods

Optimization methods Lecture notes 3 February 8, 016 1 Introduction Optimization methods In these notes we provide an overview of a selection of optimization methods. We focus on methods which rely on first-order information,

More information

Multiple Facilities Location in the Plane Using the Gravity Model

Multiple Facilities Location in the Plane Using the Gravity Model Geographical Analysis ISSN 0016-7363 Multiple Facilities Location in the Plane Using the Gravity Model Tammy Drezner, Zvi Drezner College of Business and Economics, California State University-Fullerton,

More information

Subgradient. Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes. definition. subgradient calculus

Subgradient. Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes. definition. subgradient calculus 1/41 Subgradient Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes definition subgradient calculus duality and optimality conditions directional derivative Basic inequality

More information

Support Vector Machine

Support Vector Machine Support Vector Machine Fabrice Rossi SAMM Université Paris 1 Panthéon Sorbonne 2018 Outline Linear Support Vector Machine Kernelized SVM Kernels 2 From ERM to RLM Empirical Risk Minimization in the binary

More information

Support Vector Machines Explained

Support Vector Machines Explained December 23, 2008 Support Vector Machines Explained Tristan Fletcher www.cs.ucl.ac.uk/staff/t.fletcher/ Introduction This document has been written in an attempt to make the Support Vector Machines (SVM),

More information

NONLINEAR. (Hillier & Lieberman Introduction to Operations Research, 8 th edition)

NONLINEAR. (Hillier & Lieberman Introduction to Operations Research, 8 th edition) NONLINEAR PROGRAMMING (Hillier & Lieberman Introduction to Operations Research, 8 th edition) Nonlinear Programming g Linear programming has a fundamental role in OR. In linear programming all its functions

More information

Facility Location Problems with Random Demands in a Competitive Environment

Facility Location Problems with Random Demands in a Competitive Environment Facility Location Problems with Random Demands in a Competitive Environment Takeshi Uno, Hideki Katagiri, and Kosuke Kato Abstract This paper proposes a new location problem of competitive facilities,

More information

APPROXIMATE METHODS FOR CONVEX MINIMIZATION PROBLEMS WITH SERIES PARALLEL STRUCTURE

APPROXIMATE METHODS FOR CONVEX MINIMIZATION PROBLEMS WITH SERIES PARALLEL STRUCTURE APPROXIMATE METHODS FOR CONVEX MINIMIZATION PROBLEMS WITH SERIES PARALLEL STRUCTURE ADI BEN-ISRAEL, GENRIKH LEVIN, YURI LEVIN, AND BORIS ROZIN Abstract. Consider a problem of minimizing a separable, strictly

More information

EE613 Machine Learning for Engineers. Kernel methods Support Vector Machines. jean-marc odobez 2015

EE613 Machine Learning for Engineers. Kernel methods Support Vector Machines. jean-marc odobez 2015 EE613 Machine Learning for Engineers Kernel methods Support Vector Machines jean-marc odobez 2015 overview Kernel methods introductions and main elements defining kernels Kernelization of k-nn, K-Means,

More information

Generalization to inequality constrained problem. Maximize

Generalization to inequality constrained problem. Maximize Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum

More information

Convex envelopes, cardinality constrained optimization and LASSO. An application in supervised learning: support vector machines (SVMs)

Convex envelopes, cardinality constrained optimization and LASSO. An application in supervised learning: support vector machines (SVMs) ORF 523 Lecture 8 Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Any typos should be emailed to a a a@princeton.edu. 1 Outline Convexity-preserving operations Convex envelopes, cardinality

More information

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 18

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 18 EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 18 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 31, 2012 Andre Tkacenko

More information

DO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO

DO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO QUESTION BOOKLET EECS 227A Fall 2009 Midterm Tuesday, Ocotober 20, 11:10-12:30pm DO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO You have 80 minutes to complete the midterm. The midterm consists

More information

Support Vector Machine via Nonlinear Rescaling Method

Support Vector Machine via Nonlinear Rescaling Method Manuscript Click here to download Manuscript: svm-nrm_3.tex Support Vector Machine via Nonlinear Rescaling Method Roman Polyak Department of SEOR and Department of Mathematical Sciences George Mason University

More information

The Relation Between Pseudonormality and Quasiregularity in Constrained Optimization 1

The Relation Between Pseudonormality and Quasiregularity in Constrained Optimization 1 October 2003 The Relation Between Pseudonormality and Quasiregularity in Constrained Optimization 1 by Asuman E. Ozdaglar and Dimitri P. Bertsekas 2 Abstract We consider optimization problems with equality,

More information

Decision Science Letters

Decision Science Letters Decision Science Letters 8 (2019) *** *** Contents lists available at GrowingScience Decision Science Letters homepage: www.growingscience.com/dsl A new logarithmic penalty function approach for nonlinear

More information

Unsupervised Learning Techniques Class 07, 1 March 2006 Andrea Caponnetto

Unsupervised Learning Techniques Class 07, 1 March 2006 Andrea Caponnetto Unsupervised Learning Techniques 9.520 Class 07, 1 March 2006 Andrea Caponnetto About this class Goal To introduce some methods for unsupervised learning: Gaussian Mixtures, K-Means, ISOMAP, HLLE, Laplacian

More information

Heuristics for The Whitehead Minimization Problem

Heuristics for The Whitehead Minimization Problem Heuristics for The Whitehead Minimization Problem R.M. Haralick, A.D. Miasnikov and A.G. Myasnikov November 11, 2004 Abstract In this paper we discuss several heuristic strategies which allow one to solve

More information

Complexity of local search for the p-median problem

Complexity of local search for the p-median problem Complexity of local search for the p-median problem Ekaterina Alekseeva, Yuri Kochetov, Alexander Plyasunov Sobolev Institute of Mathematics, Novosibirsk {ekaterina2, jkochet, apljas}@math.nsc.ru Abstract

More information

A Robust APTAS for the Classical Bin Packing Problem

A Robust APTAS for the Classical Bin Packing Problem A Robust APTAS for the Classical Bin Packing Problem Leah Epstein 1 and Asaf Levin 2 1 Department of Mathematics, University of Haifa, 31905 Haifa, Israel. Email: lea@math.haifa.ac.il 2 Department of Statistics,

More information

CS6375: Machine Learning Gautam Kunapuli. Support Vector Machines

CS6375: Machine Learning Gautam Kunapuli. Support Vector Machines Gautam Kunapuli Example: Text Categorization Example: Develop a model to classify news stories into various categories based on their content. sports politics Use the bag-of-words representation for this

More information

Math 273a: Optimization Subgradients of convex functions

Math 273a: Optimization Subgradients of convex functions Math 273a: Optimization Subgradients of convex functions Made by: Damek Davis Edited by Wotao Yin Department of Mathematics, UCLA Fall 2015 online discussions on piazza.com 1 / 42 Subgradients Assumptions

More information

1 Column Generation and the Cutting Stock Problem

1 Column Generation and the Cutting Stock Problem 1 Column Generation and the Cutting Stock Problem In the linear programming approach to the traveling salesman problem we used the cutting plane approach. The cutting plane approach is appropriate when

More information

many authors. For an overview see e.g. [5, 6, 3, 6]. In practice the modeling of the investigated region as the complete IR is not realistic. There ma

many authors. For an overview see e.g. [5, 6, 3, 6]. In practice the modeling of the investigated region as the complete IR is not realistic. There ma Planar Weber Location Problems with Line Barriers K. Klamroth Λ Universität Kaiserslautern Abstract The Weber problem for a given finite set of existing facilities in the plane is to find the location

More information

Support Vector Machines for Classification: A Statistical Portrait

Support Vector Machines for Classification: A Statistical Portrait Support Vector Machines for Classification: A Statistical Portrait Yoonkyung Lee Department of Statistics The Ohio State University May 27, 2011 The Spring Conference of Korean Statistical Society KAIST,

More information

3.10 Lagrangian relaxation

3.10 Lagrangian relaxation 3.10 Lagrangian relaxation Consider a generic ILP problem min {c t x : Ax b, Dx d, x Z n } with integer coefficients. Suppose Dx d are the complicating constraints. Often the linear relaxation and the

More information

Convex Optimization. Problem set 2. Due Monday April 26th

Convex Optimization. Problem set 2. Due Monday April 26th Convex Optimization Problem set 2 Due Monday April 26th 1 Gradient Decent without Line-search In this problem we will consider gradient descent with predetermined step sizes. That is, instead of determining

More information

Optimization and Gradient Descent

Optimization and Gradient Descent Optimization and Gradient Descent INFO-4604, Applied Machine Learning University of Colorado Boulder September 12, 2017 Prof. Michael Paul Prediction Functions Remember: a prediction function is the function

More information

Chapter 1 - Preference and choice

Chapter 1 - Preference and choice http://selod.ensae.net/m1 Paris School of Economics (selod@ens.fr) September 27, 2007 Notations Consider an individual (agent) facing a choice set X. Definition (Choice set, "Consumption set") X is a set

More information

A PREDICTOR-CORRECTOR PATH-FOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE

A PREDICTOR-CORRECTOR PATH-FOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE Yugoslav Journal of Operations Research 24 (2014) Number 1, 35-51 DOI: 10.2298/YJOR120904016K A PREDICTOR-CORRECTOR PATH-FOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE BEHROUZ

More information

A Brief Review on Convex Optimization

A Brief Review on Convex Optimization A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review

More information

FUZZY C-MEANS CLUSTERING USING TRANSFORMATIONS INTO HIGH DIMENSIONAL SPACES

FUZZY C-MEANS CLUSTERING USING TRANSFORMATIONS INTO HIGH DIMENSIONAL SPACES FUZZY C-MEANS CLUSTERING USING TRANSFORMATIONS INTO HIGH DIMENSIONAL SPACES Sadaaki Miyamoto Institute of Engineering Mechanics and Systems University of Tsukuba Ibaraki 305-8573, Japan Daisuke Suizu Graduate

More information

Lecture 9: Large Margin Classifiers. Linear Support Vector Machines

Lecture 9: Large Margin Classifiers. Linear Support Vector Machines Lecture 9: Large Margin Classifiers. Linear Support Vector Machines Perceptrons Definition Perceptron learning rule Convergence Margin & max margin classifiers (Linear) support vector machines Formulation

More information

Finding normalized and modularity cuts by spectral clustering. Ljubjana 2010, October

Finding normalized and modularity cuts by spectral clustering. Ljubjana 2010, October Finding normalized and modularity cuts by spectral clustering Marianna Bolla Institute of Mathematics Budapest University of Technology and Economics marib@math.bme.hu Ljubjana 2010, October Outline Find

More information

c 4, < y 2, 1 0, otherwise,

c 4, < y 2, 1 0, otherwise, Fundamentals of Big Data Analytics Univ.-Prof. Dr. rer. nat. Rudolf Mathar Problem. Probability theory: The outcome of an experiment is described by three events A, B and C. The probabilities Pr(A) =,

More information

Mirror Descent for Metric Learning. Gautam Kunapuli Jude W. Shavlik

Mirror Descent for Metric Learning. Gautam Kunapuli Jude W. Shavlik Mirror Descent for Metric Learning Gautam Kunapuli Jude W. Shavlik And what do we have here? We have a metric learning algorithm that uses composite mirror descent (COMID): Unifying framework for metric

More information

8. Geometric problems

8. Geometric problems 8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 Minimum volume ellipsoid around a set Löwner-John ellipsoid

More information

An algorithm to increase the node-connectivity of a digraph by one

An algorithm to increase the node-connectivity of a digraph by one Discrete Optimization 5 (2008) 677 684 Contents lists available at ScienceDirect Discrete Optimization journal homepage: www.elsevier.com/locate/disopt An algorithm to increase the node-connectivity of

More information

Support Vector Machine (continued)

Support Vector Machine (continued) Support Vector Machine continued) Overlapping class distribution: In practice the class-conditional distributions may overlap, so that the training data points are no longer linearly separable. We need

More information

Mathematics 530. Practice Problems. n + 1 }

Mathematics 530. Practice Problems. n + 1 } Department of Mathematical Sciences University of Delaware Prof. T. Angell October 19, 2015 Mathematics 530 Practice Problems 1. Recall that an indifference relation on a partially ordered set is defined

More information

Homework 2 Solutions Kernel SVM and Perceptron

Homework 2 Solutions Kernel SVM and Perceptron Homework 2 Solutions Kernel SVM and Perceptron CMU 1-71: Machine Learning (Fall 21) https://piazza.com/cmu/fall21/17115781/home OUT: Sept 25, 21 DUE: Oct 8, 11:59 PM Problem 1: SVM decision boundaries

More information

Improved Algorithms for Machine Allocation in Manufacturing Systems

Improved Algorithms for Machine Allocation in Manufacturing Systems Improved Algorithms for Machine Allocation in Manufacturing Systems Hans Frenk Martine Labbé Mario van Vliet Shuzhong Zhang October, 1992 Econometric Institute, Erasmus University Rotterdam, the Netherlands.

More information

arxiv: v1 [math.oc] 1 Jul 2016

arxiv: v1 [math.oc] 1 Jul 2016 Convergence Rate of Frank-Wolfe for Non-Convex Objectives Simon Lacoste-Julien INRIA - SIERRA team ENS, Paris June 8, 016 Abstract arxiv:1607.00345v1 [math.oc] 1 Jul 016 We give a simple proof that the

More information