9. Geometric problems

Transcription

1 9. Geometric problems EE/AA 578, Univ of Washington, Fall 2016 projection on a set extremal volume ellipsoids centering classification 9 1

2 Projection on convex set projection of point x on set C defined as P C (x) =argmin z C x z i.e., point in C closest to x. suppose C has form f i : R n R convex C = { x Ax = b, f i (x) 0, i =1,...,m} P C (x) = argmin x z subject to Az = b f i (z) 0, i =1,...,m computing P C (x) is cvx opt problem 10 2

3 Distance between convex sets distance between sets C, C defined as dist(c, C) = min z C, z C z z suppose C, C are convex, with form f i, f i : R n R convex C = { x Ax = b, f i (x) 0, i =1,...,m} C = { x Ãx = b, fi (x) 0, i =1,..., m } dist(c, C) is optimal value of cvx problem minimize z z subject to Az = b, Ã z = b f i (z) 0, i =1,...,m f i ( z) 0, i =1,..., m 10 3

4 Intersection & containment of polyhedra inequality description P 1 = {x a T i x b i,i=1,...,m} = {x Ax b} P 2 = {x fi T x g i,i=1,...,l} = {x Fx g} P 1 P 2 =? solve feasibility problem Ax b, F x g P 1 P 2?fork =1,...,l,check sup{f T k x Ax b } g k, i.e., solve LPs maximize subject to fk T x Ax b 10 4

5 Minimum volume ellipsoid around a set Löwner-John ellipsoid of a set C: minimum volume ellipsoid E s.t. C E parametrize E as E = {v Av +b 2 1}; w.l.o.g. assume A S n ++ vole is proportional to deta 1 ; to compute minimum volume ellipsoid, minimize (over A, /Users/maryamfazel/Desktop/geom u pdated(dragged).p subject to convex, but evaluating the constraint can be hard (for general C) finite set C = {x 1,...,x m }: minimize (over A, b) logdeta 1 subject to Ax i +b 2 1, i = 1,...,m also gives Löwner-John ellipsoid for polyhedron conv{x 1,...,x m } 9 2

6 Maximum volume inscribed ellipsoid maximum volume ellipsoid E inside a convex set C R n parametrize E as E = {Bu+d u 2 1}; w.l.o.g. assume B S n ++ vole is proportional to detb; can compute E by solving maximize log det B subject to sup u 2 1I C (Bu+d) 0 (where I C (x) = 0 for x C and I C (x) = for x C) convex, but evaluating the constraint can be hard (for general C) polyhedron {x a T i x b i, i = 1,...,m}: maximize log det B subject to Ba i 2 +a T i d b i, i = 1,...,m (constraint follows from sup u 2 1a T i (Bu+d) = Ba i 2 +a T i d) 9 3

7 Efficiency of ellipsoidal approximations C R n convex, bounded, with nonempty interior Löwner-John ellipsoid, shrunk by a factor n, lies inside C maximum volume inscribed ellipsoid, expanded by a factor n, covers C example (for two polyhedra in R 2 ) factor n can be improved to n if C is symmetric 9 4

8 Centering some possible definitions of center of a convex set C: center of largest inscribed ball ( Chebyshev center ) for polyhedron, can be computed via linear programming (see LP lecture) center of maximum volume inscribed ellipsoid (page 9 3) x cheb x mve 9 5

9 Linear discrimination separate two sets of points {x 1,...,x N }, {y 1,...,y M } by a hyperplane: a T x i +b > 0, i = 1,...,N, a T y i +b < 0, i = 1,...,M homogeneous in a, b, hence equivalent to a T x i +b 1, i = 1,...,N, a T y i +b 1, i = 1,...,M a set of linear inequalities in a, b 9 6

10 Robust linear discrimination (Euclidean) distance between hyperplanes H 1 = {z a T z +b = 1} H 2 = {z a T z +b = 1} is dist(h 1,H 2 ) = 2/ a 2 to separate two sets of points by maximum margin, minimize (1/2) a 2 subject to a T x i +b 1, i = 1,...,N a T y i +b 1, i = 1,...,M (1) (after squaring objective) a QP in a, b 9 7

11 Lagrange dual of maximum margin separation problem (1) maximize 1 T λ+1 T µ subject to 2 N i=1 λ ix i M i=1 µ 2 iy i 1 1 T λ = 1 T µ, λ 0, µ 0 (2) from duality, optimal value is inverse of maximum margin of separation interpretation change variables to θ i = λ i /1 T λ, γ i = µ i /1 T µ, t = 1/(1 T λ+1 T µ) invert objective to minimize 1/(1 T λ+1 T µ) = t minimize t subject to N i=1 θ ix i M i=1 γ 2 iy i t θ 0, 1 T θ = 1, γ 0, 1 T γ = 1 optimal value is distance between convex hulls 9 8

12 Approximate linear separation of non-separable sets an LP in a, b, u, v minimize 1 T u+1 T v subject to a T x i +b 1 u i, i = 1,...,N a T y i +b 1+v i, i = 1,...,M u 0, v 0 at optimum, u i = max{0,1 a T x i b}, v i = max{0,1+a T y i +b} can be interpreted as a heuristic for minimizing #misclassified points 9 9

13 Support vector classifier minimize a 2 +γ(1 T u+1 T v) subject to a T x i +b 1 u i, i = 1,...,N a T y i +b 1+v i, i = 1,...,M u 0, v 0 produces point on trade-off curve between inverse of margin 2/ a 2 and classification error, measured by total slack 1 T u+1 T v same example as previous page, with γ = 0.1: 9 10

14 Nonlinear discrimination separate two sets of points by a nonlinear function: f(x i ) > 0, i = 1,...,N, f(y i ) < 0, i = 1,...,M choose a linearly parametrized family of functions f(z) = θ T F(z) F = (F 1,...,F k ) : R n R k are basis functions solve a set of linear inequalities in θ: θ T F(x i ) 1, i = 1,...,N, θ T F(y i ) 1, i = 1,...,M 9 11

15 quadratic discrimination: f(z) = z T Pz +q T z +r x T i Px i +q T x i +r 1, y T i Py i +q T y i +r 1 can add additional constraints (e.g., P I to separate by an ellipsoid) polynomial discrimination: F(z) are all monomials up to a given degree separation by ellipsoid separation by 4th degree polynomial 9 12

16 Placement and facility location N points with coordinates x i R 2 (or R 3 ) some positions x i are given; the other x i s are variables for each pair of points, a cost function f ij (x i,x j ) placement problem minimize i j f ij(x i,x j ) variables are positions of free points interpretations points represent plants or warehouses; f ij is transportation cost between facilities i and j points represent cells on an IC; f ij represents wirelength 9 13

17 example: minimize (i,j) A h( x i x j 2 ), with 6 free points, 27 links optimal placement for h(z) = z, h(z) = z 2, h(z) = z histograms of connection lengths x i x j