Combinatorial Optimization: AMS Amitabh Basu. Department of Applied Mathematics and Statistics, Johns Hopkins U.

Size: px

Start display at page:

Download "Combinatorial Optimization: AMS Amitabh Basu. Department of Applied Mathematics and Statistics, Johns Hopkins U."

Bruno Powers
6 years ago
Views:

1 Combinatorial Optimization: AMS Amitabh Basu Department of Applied Mathematics and Statistics, Johns Hopkins U., Spring / 7

2 Two types of optimization problems Type I n Items: weights w 1,..., w n, values: c 1,..., c n Knapsack with capacity W. Find the subset with most value that can fit into the knapsack. Type II n Food Items: Each item has price/cost p i per unit weight, nutrition value per unit weight (fat, carbohydrates, vitamins etc.). Find right combination of food items (weights in pounds) with least cost that meets all nutritional requirements. 2 / 7

3 Two types of optimization problems Type I n Items: weights w 1,..., w n, values: c 1,..., c n Knapsack with capacity W. Find the subset with most value that can fit into the knapsack. Type II n Food Items: Each item has price/cost p i per unit weight, nutrition value per unit weight (fat, carbohydrates, vitamins etc.). Find right combination of food items (weights in pounds) with least cost that meets all nutritional requirements. 1. Brute force approach for Type I does not scale. 2. Classical techniques available for Type II: Calculus, convexity - Does not apply to Type I 2 / 7

4 Two types of optimization problems Type I Combinatorial Optimization Type II Continuous Optimization n Items: weights w 1,..., w n, values: c 1,..., c n Knapsack with capacity W. Find the subset with most value that can fit into the knapsack. n Food Items: Each item has price/cost p i per unit weight, nutrition value per unit weight (fat, carbohydrates, vitamins etc.). Find right combination of food items (weights in pounds) with least cost that meets all nutritional requirements. 1. Brute force approach for Type I does not scale. 2. Classical techniques available for Type II: Calculus, convexity - Does not apply to Type I 2 / 7

5 A Transportation Problem 3 / 7

6 A Simpler Problem - Transshipment problem / 7

7 A Scheduling Problem Job 1 Job 2 Job 3 Job 4 Machine 1 Machine 2 Machine 3 Machine 4 Machine 5 5 / 7

8 A Problem from Astronomy 6 / 7

9 A Problem from Astronomy 6 / 7

10 A Problem from Astronomy Use physics to derive an evaluation function that evaluates a given partition (Correlation function in astronomy) 6 / 7

11 A Problem from Astronomy Use physics to derive an evaluation function that evaluates a given partition (Correlation function in astronomy) 6 / 7

12 A Problem from Astronomy Use physics to derive an evaluation function that evaluates a given partition (Correlation function in astronomy) 6 / 7

13 A Problem from Astronomy 1000 galaxies: possible partitions Evaluate a partition in seconds Will take years!!!! Use physics to derive an evaluation function that evaluates a given partition (Correlation function in astronomy) 6 / 7

14 Statistical/Machine Learning Linear regression: Given a bunch of points x 1,..., x D R n, and labels y 1,..., y D R, find the best linear function that fits this data. 7 / 7

15 Statistical/Machine Learning Linear regression: Given a bunch of points x 1,..., x D R n, and labels y 1,..., y D R, find the best linear function that fits this data. min β R n D (y i β T x i ) 2 i=1 7 / 7

16 Statistical/Machine Learning Linear regression: Given a bunch of points x 1,..., x D R n, and labels y 1,..., y D R, find the best linear function that fits this data. min β R n D (y i β T x i ) 2 i=1 Better statistical guarantees if we enforce sparsity on β. 7 / 7

17 Statistical/Machine Learning Linear regression: Given a bunch of points x 1,..., x D R n, and labels y 1,..., y D R, find the best linear function that fits this data. min β R n D (y i β T x i ) 2 i=1 subject to β 0 K Better statistical guarantees if we enforce sparsity on β. 7 / 7

18 Statistical/Machine Learning Linear regression: Given a bunch of points x 1,..., x D R n, and labels y 1,..., y D R, find the best linear function that fits this data. min β R n D (y i β T x i ) 2 i=1 subject to β 0 K Better statistical guarantees if we enforce sparsity on β. Best Subset Selection via a Modern Optimization Lens by Bertsimas, King, Mazumder in Annals of Statistics / 7

19 Statistical/Machine Learning Linear regression: Given a bunch of points x 1,..., x D R n, and labels y 1,..., y D R, find the best linear function that fits this data. min β R n D (y i β T x i ) 2 i=1 subject to β 0 K Better statistical guarantees if we enforce sparsity on β. Best Subset Selection via a Modern Optimization Lens by Bertsimas, King, Mazumder in Annals of Statistics 2017 Similar problem in Compressed Sensing or Sparse Coding. 7 / 7

MLCC 2018 Variable Selection and Sparsity. Lorenzo Rosasco UNIGE-MIT-IIT

MLCC 2018 Variable Selection and Sparsity Lorenzo Rosasco UNIGE-MIT-IIT Outline Variable Selection Subset Selection Greedy Methods: (Orthogonal) Matching Pursuit Convex Relaxation: LASSO & Elastic Net