Lecture Note 1: Introduction to optimization. Xiaoqun Zhang Shanghai Jiao Tong University
|
|
- Nathaniel Daniel
- 5 years ago
- Views:
Transcription
1 Lecture Note 1: Introduction to optimization Xiaoqun Zhang Shanghai Jiao Tong University Last updated: September 23, 2017
2 1.1 Introduction 1. Optimization is an important tool in daily life, business and engineer: Running a business: to maximize profit, minimize loss, maximize efficiency, or minimize risk. Design: minimize the weight of a bridge, and maximize the strength, within the design constrains; airplane engineering such as shape design and material selection; packing transistors in a computer chip in a functional way. Planning: select a flight route to minimize time or fuel consumption of an airplane Supermarket pricing and logistic 2. Formal definition: to minimize (or maximize) a real function by deciding the values of free variables from within an allowed set. 3. Status of optimization: in last few decades, astonishing improvements in computer hardware and software motivated optimization modeling, algorithm designs, and implementation. Solving certain optimization problems has become standard techniques and everyday practice in businesses, science and engineering. It is now possible to solve certain optimization problems with thousands, millions and even thousands of millions of variables; optimization (along with statistics) has been the foundation of machine learning and big-data analytic. 4. Ingredients of successful optimization: modeling (turn a problem into one of the typical optimization formulations); algorithms an (iterative) procedure that leads you toward a solution (most optimization problems do not have a closed-form solution); software and hardware implementation: realize the algorithms and return numerical solutions. 5. Optimization formulation: mininize x (maximize) f(x) subject to x C minimize is often abbreviated as min ; decision variable is typically stated under minimize, unless obvious; subject to is often shorten as s.t. ; in linear and nonlinear optimization, feasible set C is represented by h i (x) = b i, i E(equality constraints) g j (x) b j, j I(inequality constrains) 2
3 6. First examples: An office furniture manufacturer wants to maximize the daily profit. Decide the daily number of desks and chairs that should be manufactured such that the profit is maximum. The required material and profit of manufacturer desks and chair is listed as followed: Each piece Required material Required labor Profit desk chair Total available Problem: let the daily manufactured number of desks and chairs be x 1, x 2, solve max x 1,x 2 20x 1 +10x 2 s.t. 10x 1 +6x x 1 +2x x 1,x 2 0 Find two nonnegative numbers whosesumis up to6sothattheir product is a maximum. ( find the largest area of a rectangular region provided that its perimeter is no greater than 12). Problem: let the two numbers be x,y, solve max x,y xy s.t. x+y = 6 x,y 0 Find a line that best fit three given points (x 1,y 1 ) = (2,1),(x 2,y 2 ) = (3,6) and (x 3,y 3 ) = (5,4). Problem: let the line equation be y = ax+b, in the following least square sense: min a,b 3 (ax i +b y i ) 2 = (2a+b 1) 2 +(3a+b 6) 2 +(5a+b 4) 2 i=1 Traveling sales man problem: A salesman needs to visit a number (n) of cities, denoted by city 1,2,,n. The distances between cities are known, i.e. d ij denotes the distance between city i and city j. The salesman wants to travel each city once in turn and in the meantime 3
4 minimize the total travel distance. What order should be used? Problem: denote the order of travelling by (i 1,,i n ), model n 1 min d ik,i k+1 k=1 s.t. (i 1,i 2,,i n ) is a permutation of (1,2,,n) 7. Classification of optimization problems 8. Convex programming: Continuous vs Discrete Constrained vs Unconstrained Global vs Local Stochastic vs Deterministic Convex vs Nonconvex minf(x) s.t. X C both f and C are convex. Special cases: linear programming minc T x s.t. Ax b;bx = d; least square problems (quadratic program): minx T Qx+c T x s.t. Ax b;bx = d; 9. Global vs local solution: solutions means optimal solution. Global solution x : f(x ) f(x) for all x C. Local solutions x : δ > 0 such that f(x ) f(x) for all x C and x x δ. A (global or local) solution x is unique if holds strictly as <. In general, it is difficult to tell if a local solution is global because algorithms can only check nearby points and have not clue of behaviors father way. Hence a solution may refer to a local solution. A local solution to a convexprogram if globally optimal. A LP is convex. A stationary point (where the derivatives is zero is also known as a solution, but it can be a maximization, minimization 10. Nonlinear program: min f(x) = (x 1 +x 2 ) 2 s.t.x 1 x x x 2 1 Decision variables, feasible set, objective, (box) constraints,global minimizer (x = ( 2, 2)) vs local minimizer (x = (1,1)). See the figure 10. 4
5 3 2 A 1 H D x (Local solution 4 3 F 2 1 E I B 2 x g G (Global solution) C Figure 1.1: Global solution vs. local solution 11. Optimization algorithms requirement Robustness: perform well on a wide variety of problem Efficiency: reasonable computer time or storage Accuracy: precision, not being sensitive to norms Using convex optimization problem: often difficult to recognize, many tricks for transforming problems into convex form, surprisingly many problems can be solved via convex optimization. 12. This course focuses on finding local solutions and, for convex programs, global solutions. Asking for global solutions is computationally intractable, in general. Many useful problems are convex, that is, a local solution is global In some applications, a local solution is an improvement from an existing point. Local solutions are OK. 13. We do not cover discrete optimization: Discrete variables often take binary or integer values, or values from a discrete set. Those problems are called discrete optimization. 5
Math 273a: Optimization
Math 273a: Optimization Instructor: Wotao Yin Department of Mathematics, UCLA Fall 2015 online discussions on piazza.com What is mathematical optimization? Optimization models the goal of solving a problem
More informationNumerical Optimization. Review: Unconstrained Optimization
Numerical Optimization Finding the best feasible solution Edward P. Gatzke Department of Chemical Engineering University of South Carolina Ed Gatzke (USC CHE ) Numerical Optimization ECHE 589, Spring 2011
More informationLecture 3: Basics of set-constrained and unconstrained optimization
Lecture 3: Basics of set-constrained and unconstrained optimization (Chap 6 from textbook) Xiaoqun Zhang Shanghai Jiao Tong University Last updated: October 9, 2018 Optimization basics Outline Optimization
More informationAnalysis of Algorithms. Unit 5 - Intractable Problems
Analysis of Algorithms Unit 5 - Intractable Problems 1 Intractable Problems Tractable Problems vs. Intractable Problems Polynomial Problems NP Problems NP Complete and NP Hard Problems 2 In this unit we
More informationMathematical Optimization Models and Applications
Mathematical Optimization Models and Applications Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 1, 2.1-2,
More informationOptimization Models and Applications
Optimization Models and Applications Martin Takáč ISE 316, Fall 2014, Lecture 1 September 3, 2014 Martin Takáč ISE 316 Fall 2014 1 / 33 Outline Course Information Example of an Optimization Problem Terminology
More informationWhat is an integer program? Modelling with Integer Variables. Mixed Integer Program. Let us start with a linear program: max cx s.t.
Modelling with Integer Variables jesla@mandtudk Department of Management Engineering Technical University of Denmark What is an integer program? Let us start with a linear program: st Ax b x 0 where A
More informationConvex Optimization & Machine Learning. Introduction to Optimization
Convex Optimization & Machine Learning Introduction to Optimization mcuturi@i.kyoto-u.ac.jp CO&ML 1 Why do we need optimization in machine learning We want to find the best possible decision w.r.t. a problem
More informationAlgorithms & Data Structures 2
Algorithms & Data Structures 2 Optimization WS2017 B. Anzengruber-Tanase (Institute for Pervasive Computing, JKU Linz) (Institute for Pervasive Computing, JKU Linz) WHAT IS OPTIMIZATION? Optimization problems
More informationSupport Vector Machines: Maximum Margin Classifiers
Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and Fu-Jie Huang 1 Outline What is behind
More informationLecture 18: Kernels Risk and Loss Support Vector Regression. Aykut Erdem December 2016 Hacettepe University
Lecture 18: Kernels Risk and Loss Support Vector Regression Aykut Erdem December 2016 Hacettepe University Administrative We will have a make-up lecture on next Saturday December 24, 2016 Presentations
More information36106 Managerial Decision Modeling Linear Decision Models: Part II
1 36106 Managerial Decision Modeling Linear Decision Models: Part II Kipp Martin University of Chicago Booth School of Business January 20, 2014 Reading and Excel Files Reading (Powell and Baker): Sections
More informationChapter 3: Discrete Optimization Integer Programming
Chapter 3: Discrete Optimization Integer Programming Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Sito web: http://home.deib.polimi.it/amaldi/ott-13-14.shtml A.A. 2013-14 Edoardo
More informationMath 381 Midterm Practice Problem Solutions
Math 381 Midterm Practice Problem Solutions Notes: -Many of the exercises below are adapted from Operations Research: Applications and Algorithms by Winston. -I have included a list of topics covered on
More informationIn complexity theory, algorithms and problems are classified by the growth order of computation time as a function of instance size.
10 2.2. CLASSES OF COMPUTATIONAL COMPLEXITY An optimization problem is defined as a class of similar problems with different input parameters. Each individual case with fixed parameter values is called
More informationHandout 1: Introduction to Dynamic Programming. 1 Dynamic Programming: Introduction and Examples
SEEM 3470: Dynamic Optimization and Applications 2013 14 Second Term Handout 1: Introduction to Dynamic Programming Instructor: Shiqian Ma January 6, 2014 Suggested Reading: Sections 1.1 1.5 of Chapter
More informationOperations Scheduling for the LSST
Operations Scheduling for the LSST Robert J. Vanderbei 2015 March 18 http://www.princeton.edu/ rvdb LSST Scheduler Workshop Tucson AZ Simple Optimization Problem Unconstrained max x R n f(x) The function
More informationOPTIMIZATION. joint course with. Ottimizzazione Discreta and Complementi di R.O. Edoardo Amaldi. DEIB Politecnico di Milano
OPTIMIZATION joint course with Ottimizzazione Discreta and Complementi di R.O. Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-15-16.shtml
More informationDeterministic Operations Research, ME 366Q and ORI 391 Chapter 2: Homework #2 Solutions
Deterministic Operations Research, ME 366Q and ORI 391 Chapter 2: Homework #2 Solutions 11. Consider the following linear program. Maximize z = 6x 1 + 3x 2 subject to x 1 + 2x 2 2x 1 + x 2 20 x 1 x 2 x
More informationThe Method of Substitution. Linear and Nonlinear Systems of Equations. The Method of Substitution. The Method of Substitution. Example 2.
The Method of Substitution Linear and Nonlinear Systems of Equations Precalculus 7.1 Here is an example of a system of two equations in two unknowns. Equation 1 x + y = 5 Equation 3x y = 4 A solution of
More informationJanuary 29, Introduction to optimization and complexity. Outline. Introduction. Problem formulation. Convexity reminder. Optimality Conditions
Olga Galinina olga.galinina@tut.fi ELT-53656 Network Analysis Dimensioning II Department of Electronics Communications Engineering Tampere University of Technology, Tampere, Finl January 29, 2014 1 2 3
More informationLecture 1. Stochastic Optimization: Introduction. January 8, 2018
Lecture 1 Stochastic Optimization: Introduction January 8, 2018 Optimization Concerned with mininmization/maximization of mathematical functions Often subject to constraints Euler (1707-1783): Nothing
More informationChapter 3: Discrete Optimization Integer Programming
Chapter 3: Discrete Optimization Integer Programming Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-16-17.shtml Academic year 2016-17
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationGestion de la production. Book: Linear Programming, Vasek Chvatal, McGill University, W.H. Freeman and Company, New York, USA
Gestion de la production Book: Linear Programming, Vasek Chvatal, McGill University, W.H. Freeman and Company, New York, USA 1 Contents 1 Integer Linear Programming 3 1.1 Definitions and notations......................................
More informationSECTION 5.1: Polynomials
1 SECTION 5.1: Polynomials Functions Definitions: Function, Independent Variable, Dependent Variable, Domain, and Range A function is a rule that assigns to each input value x exactly output value y =
More informationSupport Vector Machines and Kernel Methods
2018 CS420 Machine Learning, Lecture 3 Hangout from Prof. Andrew Ng. http://cs229.stanford.edu/notes/cs229-notes3.pdf Support Vector Machines and Kernel Methods Weinan Zhang Shanghai Jiao Tong University
More information56:171 Operations Research Midterm Exam - October 26, 1989 Instructor: D.L. Bricker
56:171 Operations Research Midterm Exam - October 26, 1989 Instructor: D.L. Bricker Answer all of Part One and two (of the four) problems of Part Two Problem: 1 2 3 4 5 6 7 8 TOTAL Possible: 16 12 20 10
More informationConvex Optimization Lecture 16
Convex Optimization Lecture 16 Today: Projected Gradient Descent Conditional Gradient Descent Stochastic Gradient Descent Random Coordinate Descent Recall: Gradient Descent (Steepest Descent w.r.t Euclidean
More informationApplications of Linear Programming
Applications of Linear Programming lecturer: András London University of Szeged Institute of Informatics Department of Computational Optimization Lecture 9 Non-linear programming In case of LP, the goal
More informationLMI Methods in Optimal and Robust Control
LMI Methods in Optimal and Robust Control Matthew M. Peet Arizona State University Lecture 02: Optimization (Convex and Otherwise) What is Optimization? An Optimization Problem has 3 parts. x F f(x) :
More informationThe quest for finding Hamiltonian cycles
The quest for finding Hamiltonian cycles Giang Nguyen School of Mathematical Sciences University of Adelaide Travelling Salesman Problem Given a list of cities and distances between cities, what is the
More informationOptimization: an Overview
Optimization: an Overview Moritz Diehl University of Freiburg and University of Leuven (some slide material was provided by W. Bangerth and K. Mombaur) Overview of presentation Optimization: basic definitions
More informationReconnect 04 Introduction to Integer Programming
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, Reconnect 04 Introduction to Integer Programming Cynthia Phillips, Sandia National Laboratories Integer programming
More informationLECTURE 13 LECTURE OUTLINE
LECTURE 13 LECTURE OUTLINE Problem Structures Separable problems Integer/discrete problems Branch-and-bound Large sum problems Problems with many constraints Conic Programming Second Order Cone Programming
More informationIE 5531 Practice Midterm #2
IE 5531 Practice Midterm #2 Prof. John Gunnar Carlsson November 23, 2010 Problem 1: Nonlinear programming You are a songwriter who writes Top 40 style songs for the radio. Each song you write can be described
More informationAgenda today. Introduction to prescriptive modeling. Linear optimization models through three examples: Beyond linear optimization
Agenda today Introduction to prescriptive modeling Linear optimization models through three examples: 1 Production and inventory optimization 2 Distribution system design 3 Stochastic optimization Beyond
More informationUnit 2: Problem Classification and Difficulty in Optimization
Unit 2: Problem Classification and Difficulty in Optimization Learning goals Unit 2 I. What is the subject area of multiobjective decision analysis and multiobjective optimization; How does it relate to
More informationNeural Network Training
Neural Network Training Sargur Srihari Topics in Network Training 0. Neural network parameters Probabilistic problem formulation Specifying the activation and error functions for Regression Binary classification
More informationDRAFT Formulation and Analysis of Linear Programs
DRAFT Formulation and Analysis of Linear Programs Benjamin Van Roy and Kahn Mason c Benjamin Van Roy and Kahn Mason September 26, 2005 1 2 Contents 1 Introduction 7 1.1 Linear Algebra..........................
More informationSVAN 2016 Mini Course: Stochastic Convex Optimization Methods in Machine Learning
SVAN 2016 Mini Course: Stochastic Convex Optimization Methods in Machine Learning Mark Schmidt University of British Columbia, May 2016 www.cs.ubc.ca/~schmidtm/svan16 Some images from this lecture are
More informationOperations Research Lecture 6: Integer Programming
Operations Research Lecture 6: Integer Programming Notes taken by Kaiquan Xu@Business School, Nanjing University May 12th 2016 1 Integer programming (IP) formulations The integer programming (IP) is the
More informationLecture 3: Huggett s 1993 model. 1 Solving the savings problem for an individual consumer
UNIVERSITY OF WESTERN ONTARIO LONDON ONTARIO Paul Klein Office: SSC 4044 Extension: 85484 Email: pklein2@uwo.ca URL: http://paulklein.ca/newsite/teaching/619.php Economics 9619 Computational methods in
More informationLecture 23 Branch-and-Bound Algorithm. November 3, 2009
Branch-and-Bound Algorithm November 3, 2009 Outline Lecture 23 Modeling aspect: Either-Or requirement Special ILPs: Totally unimodular matrices Branch-and-Bound Algorithm Underlying idea Terminology Formal
More informationOn the Power of Robust Solutions in Two-Stage Stochastic and Adaptive Optimization Problems
MATHEMATICS OF OPERATIONS RESEARCH Vol. xx, No. x, Xxxxxxx 00x, pp. xxx xxx ISSN 0364-765X EISSN 156-5471 0x xx0x 0xxx informs DOI 10.187/moor.xxxx.xxxx c 00x INFORMS On the Power of Robust Solutions in
More informationComputational Integer Programming. Lecture 2: Modeling and Formulation. Dr. Ted Ralphs
Computational Integer Programming Lecture 2: Modeling and Formulation Dr. Ted Ralphs Computational MILP Lecture 2 1 Reading for This Lecture N&W Sections I.1.1-I.1.6 Wolsey Chapter 1 CCZ Chapter 2 Computational
More informationSection Notes 9. Midterm 2 Review. Applied Math / Engineering Sciences 121. Week of December 3, 2018
Section Notes 9 Midterm 2 Review Applied Math / Engineering Sciences 121 Week of December 3, 2018 The following list of topics is an overview of the material that was covered in the lectures and sections
More informationLinear Regression. S. Sumitra
Linear Regression S Sumitra Notations: x i : ith data point; x T : transpose of x; x ij : ith data point s jth attribute Let {(x 1, y 1 ), (x, y )(x N, y N )} be the given data, x i D and y i Y Here D
More informationThe Perceptron. Volker Tresp Summer 2016
The Perceptron Volker Tresp Summer 2016 1 Elements in Learning Tasks Collection, cleaning and preprocessing of training data Definition of a class of learning models. Often defined by the free model parameters
More informationOverfitting, Bias / Variance Analysis
Overfitting, Bias / Variance Analysis Professor Ameet Talwalkar Professor Ameet Talwalkar CS260 Machine Learning Algorithms February 8, 207 / 40 Outline Administration 2 Review of last lecture 3 Basic
More informationLecture 5: Computational Complexity
Lecture 5: Computational Complexity (3 units) Outline Computational complexity Decision problem, Classes N P and P. Polynomial reduction and Class N PC P = N P or P = N P? 1 / 22 The Goal of Computational
More informationTravelling Salesman Problem
Travelling Salesman Problem Fabio Furini November 10th, 2014 Travelling Salesman Problem 1 Outline 1 Traveling Salesman Problem Separation Travelling Salesman Problem 2 (Asymmetric) Traveling Salesman
More informationECE 307- Techniques for Engineering Decisions
ECE 307- Techniques for Engineering Decisions Lecture 4. Dualit Concepts in Linear Programming George Gross Department of Electrical and Computer Engineering Universit of Illinois at Urbana-Champaign DUALITY
More informationMath 1314 Lesson 19: Numerical Integration
Math 1314 Lesson 19: Numerical Integration For more complicated functions, we will use GeoGebra to find the definite integral. These will include functions that involve the exponential function, logarithms,
More informationLogistic Regression Trained with Different Loss Functions. Discussion
Logistic Regression Trained with Different Loss Functions Discussion CS640 Notations We restrict our discussions to the binary case. g(z) = g (z) = g(z) z h w (x) = g(wx) = + e z = g(z)( g(z)) + e wx =
More informationLogistic Regression Review Fall 2012 Recitation. September 25, 2012 TA: Selen Uguroglu
Logistic Regression Review 10-601 Fall 2012 Recitation September 25, 2012 TA: Selen Uguroglu!1 Outline Decision Theory Logistic regression Goal Loss function Inference Gradient Descent!2 Training Data
More informationECS171: Machine Learning
ECS171: Machine Learning Lecture 3: Linear Models I (LFD 3.2, 3.3) Cho-Jui Hsieh UC Davis Jan 17, 2018 Linear Regression (LFD 3.2) Regression Classification: Customer record Yes/No Regression: predicting
More informationSummary of the simplex method
MVE165/MMG630, The simplex method; degeneracy; unbounded solutions; infeasibility; starting solutions; duality; interpretation Ann-Brith Strömberg 2012 03 16 Summary of the simplex method Optimality condition:
More informationSimplex tableau CE 377K. April 2, 2015
CE 377K April 2, 2015 Review Reduced costs Basic and nonbasic variables OUTLINE Review by example: simplex method demonstration Outline Example You own a small firm producing construction materials for
More informationBilinear Programming: Applications in the Supply Chain Management
Bilinear Programming: Applications in the Supply Chain Management Artyom G. Nahapetyan Center for Applied Optimization Industrial and Systems Engineering Department University of Florida Gainesville, Florida
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationFinancial Optimization ISE 347/447. Lecture 21. Dr. Ted Ralphs
Financial Optimization ISE 347/447 Lecture 21 Dr. Ted Ralphs ISE 347/447 Lecture 21 1 Reading for This Lecture C&T Chapter 16 ISE 347/447 Lecture 21 2 Formalizing: Random Linear Optimization Consider the
More informationOn the Power of Robust Solutions in Two-Stage Stochastic and Adaptive Optimization Problems
MATHEMATICS OF OPERATIONS RESEARCH Vol. 35, No., May 010, pp. 84 305 issn 0364-765X eissn 156-5471 10 350 084 informs doi 10.187/moor.1090.0440 010 INFORMS On the Power of Robust Solutions in Two-Stage
More information16.410/413 Principles of Autonomy and Decision Making
6.4/43 Principles of Autonomy and Decision Making Lecture 8: (Mixed-Integer) Linear Programming for Vehicle Routing and Motion Planning Emilio Frazzoli Aeronautics and Astronautics Massachusetts Institute
More informationLINEAR PROGRAMMING MODULE Part 1 - Model Formulation INTRODUCTION
Name: LINEAR PROGRAMMING MODULE Part 1 - Model Formulation INTRODUCTION In general, a mathematical model is either deterministic or probabilistic. For example, the models and algorithms shown in the Graph-Optimization
More informationMachine Learning. Support Vector Machines. Manfred Huber
Machine Learning Support Vector Machines Manfred Huber 2015 1 Support Vector Machines Both logistic regression and linear discriminant analysis learn a linear discriminant function to separate the data
More informationProject Discussions: SNL/ADMM, MDP/Randomization, Quadratic Regularization, and Online Linear Programming
Project Discussions: SNL/ADMM, MDP/Randomization, Quadratic Regularization, and Online Linear Programming Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305,
More informationMAT016: Optimization
MAT016: Optimization M.El Ghami e-mail: melghami@ii.uib.no URL: http://www.ii.uib.no/ melghami/ March 29, 2011 Outline for today The Simplex method in matrix notation Managing a production facility The
More informationSection #2: Linear and Integer Programming
Section #2: Linear and Integer Programming Prof. Dr. Sven Seuken 8.3.2012 (with most slides borrowed from David Parkes) Housekeeping Game Theory homework submitted? HW-00 and HW-01 returned Feedback on
More informationMODELING (Integer Programming Examples)
MODELING (Integer Programming Eamples) IE 400 Principles of Engineering Management Integer Programming: Set 5 Integer Programming: So far, we have considered problems under the following assumptions:
More informationTheoretical questions and problems to practice Advanced Mathematics and Statistics (MSc course)
Theoretical questions and problems to practice Advanced Mathematics and Statistics (MSc course) Faculty of Business Administration M.Sc. English, 2015/16 first semester Topics (1) Complex numbers. Complex
More informationResource Constrained Project Scheduling Linear and Integer Programming (1)
DM204, 2010 SCHEDULING, TIMETABLING AND ROUTING Lecture 3 Resource Constrained Project Linear and Integer Programming (1) Marco Chiarandini Department of Mathematics & Computer Science University of Southern
More informationLINEAR PROGRAMMING: A GEOMETRIC APPROACH. Copyright Cengage Learning. All rights reserved.
3 LINEAR PROGRAMMING: A GEOMETRIC APPROACH Copyright Cengage Learning. All rights reserved. 3.4 Sensitivity Analysis Copyright Cengage Learning. All rights reserved. Sensitivity Analysis In this section,
More informationNP-Completeness. NP-Completeness 1
NP-Completeness Reference: Computers and Intractability: A Guide to the Theory of NP-Completeness by Garey and Johnson, W.H. Freeman and Company, 1979. NP-Completeness 1 General Problems, Input Size and
More informationConstrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
More information1 Strict local optimality in unconstrained optimization
ORF 53 Lecture 14 Spring 016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Thursday, April 14, 016 When in doubt on the accuracy of these notes, please cross check with the instructor s
More informationMVE165/MMG630, Applied Optimization Lecture 6 Integer linear programming: models and applications; complexity. Ann-Brith Strömberg
MVE165/MMG630, Integer linear programming: models and applications; complexity Ann-Brith Strömberg 2011 04 01 Modelling with integer variables (Ch. 13.1) Variables Linear programming (LP) uses continuous
More informationCollaborative Filtering Matrix Completion Alternating Least Squares
Case Study 4: Collaborative Filtering Collaborative Filtering Matrix Completion Alternating Least Squares Machine Learning for Big Data CSE547/STAT548, University of Washington Sham Kakade May 19, 2016
More informationInteger Programming and Branch and Bound
Courtesy of Sommer Gentry. Used with permission. Integer Programming and Branch and Bound Sommer Gentry November 4 th, 003 Adapted from slides by Eric Feron and Brian Williams, 6.40, 00. Integer Programming
More informationis called an integer programming (IP) problem. model is called a mixed integer programming (MIP)
INTEGER PROGRAMMING Integer Programming g In many problems the decision variables must have integer values. Example: assign people, machines, and vehicles to activities in integer quantities. If this is
More informationMath 211 Business Calculus TEST 3. Question 1. Section 2.2. Second Derivative Test.
Math 211 Business Calculus TEST 3 Question 1. Section 2.2. Second Derivative Test. p. 1/?? Math 211 Business Calculus TEST 3 Question 1. Section 2.2. Second Derivative Test. Question 2. Section 2.3. Graph
More informationThe use of shadow price is an example of sensitivity analysis. Duality theory can be applied to do other kind of sensitivity analysis:
Sensitivity analysis The use of shadow price is an example of sensitivity analysis. Duality theory can be applied to do other kind of sensitivity analysis: Changing the coefficient of a nonbasic variable
More informationBLITZER LEARNING GUIDE SAMPLE
BLITZER LEARNING GUIDE SAMPLE Section 5.1 Systems of Linear Equations in Two Variables Procrastination makes you sick! Researchers compared college students who were procrastinators and nonprocrastinators.
More informationLinear Regression (continued)
Linear Regression (continued) Professor Ameet Talwalkar Professor Ameet Talwalkar CS260 Machine Learning Algorithms February 6, 2017 1 / 39 Outline 1 Administration 2 Review of last lecture 3 Linear regression
More informationIntroduction to linear programming
Chapter 2 Introduction to linear programming 2.1 Single-objective optimization problem We study problems of the following form: Given a set S and a function f : S R, find, if possible, an element x S that
More informationMVE165/MMG631 Linear and integer optimization with applications Lecture 13 Overview of nonlinear programming. Ann-Brith Strömberg
MVE165/MMG631 Overview of nonlinear programming Ann-Brith Strömberg 2015 05 21 Areas of applications, examples (Ch. 9.1) Structural optimization Design of aircraft, ships, bridges, etc Decide on the material
More information1 Column Generation and the Cutting Stock Problem
1 Column Generation and the Cutting Stock Problem In the linear programming approach to the traveling salesman problem we used the cutting plane approach. The cutting plane approach is appropriate when
More informationLinear programming. Debrecen, 2015/16, 1st semester. University of Debrecen, Faculty of Business Administration 1 / 46
1 / 46 Linear programming László Losonczi University of Debrecen, Faculty of Business Administration Debrecen, 2015/16, 1st semester LP (linear programming) example 2 / 46 CMP is a cherry furniture manufacturer
More informationSTA141C: Big Data & High Performance Statistical Computing
STA141C: Big Data & High Performance Statistical Computing Lecture 8: Optimization Cho-Jui Hsieh UC Davis May 9, 2017 Optimization Numerical Optimization Numerical Optimization: min X f (X ) Can be applied
More informationA Generalized Homogeneous and Self-Dual Algorithm. for Linear Programming. February 1994 (revised December 1994)
A Generalized Homogeneous and Self-Dual Algorithm for Linear Programming Xiaojie Xu Yinyu Ye y February 994 (revised December 994) Abstract: A generalized homogeneous and self-dual (HSD) infeasible-interior-point
More informationEquations and Inequalities
Equations and Inequalities Figure 1 CHAPTER OUTLINE.1 The Rectangular Coordinate Systems and Graphs. Linear Equations in One Variable.3 Models and Applications. Comple Numbers.5 Quadratic Equations.6 Other
More informationThe P-vs-NP problem. Andrés E. Caicedo. September 10, 2011
The P-vs-NP problem Andrés E. Caicedo September 10, 2011 This note is based on lecture notes for the Caltech course Math 6c, prepared with A. Kechris and M. Shulman. 1 Decision problems Consider a finite
More informationmin f(x). (2.1) Objectives consisting of a smooth convex term plus a nonconvex regularization term;
Chapter 2 Gradient Methods The gradient method forms the foundation of all of the schemes studied in this book. We will provide several complementary perspectives on this algorithm that highlight the many
More informationEND3033 Operations Research I Sensitivity Analysis & Duality. to accompany Operations Research: Applications and Algorithms Fatih Cavdur
END3033 Operations Research I Sensitivity Analysis & Duality to accompany Operations Research: Applications and Algorithms Fatih Cavdur Introduction Consider the following problem where x 1 and x 2 corresponds
More informationMachine Learning And Applications: Supervised Learning-SVM
Machine Learning And Applications: Supervised Learning-SVM Raphaël Bournhonesque École Normale Supérieure de Lyon, Lyon, France raphael.bournhonesque@ens-lyon.fr 1 Supervised vs unsupervised learning Machine
More informationIntroduction to Optimization
Introduction to Optimization Konstantin Tretyakov (kt@ut.ee) MTAT.03.227 Machine Learning So far Machine learning is important and interesting The general concept: Fitting models to data So far Machine
More informationWarm up: risk prediction with logistic regression
Warm up: risk prediction with logistic regression Boss gives you a bunch of data on loans defaulting or not: {(x i,y i )} n i= x i 2 R d, y i 2 {, } You model the data as: P (Y = y x, w) = + exp( yw T
More informationLecture 10. Neural networks and optimization. Machine Learning and Data Mining November Nando de Freitas UBC. Nonlinear Supervised Learning
Lecture 0 Neural networks and optimization Machine Learning and Data Mining November 2009 UBC Gradient Searching for a good solution can be interpreted as looking for a minimum of some error (loss) function
More informationImproved Zero-sum Distinguisher for Full Round Keccak-f Permutation
Improved Zero-sum Distinguisher for Full Round Keccak-f Permutation Ming Duan 12 and Xuejia Lai 1 1 Department of Computer Science and Engineering, Shanghai Jiao Tong University, China. 2 Basic Courses
More informationInteger Linear Programming Modeling
DM554/DM545 Linear and Lecture 9 Integer Linear Programming Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. 2. Assignment Problem Knapsack Problem
More information