Introduction to Course

Size: px

Start display at page:

Download "Introduction to Course"

Stewart Bennett
5 years ago
Views:

1 .. Introduction to Course Oran Kittithreerapronchai 1 1 Department of Industrial Engineering, Chulalongkorn University Bangkok THAILAND last updated: September 17, 2016 COMP METH v2.00: intro 1/ 35

2 Outline. 1 Contact Information and Syllabus. 2 Roles and Agreement. 3 Motivation of Optimization. 4 Solving Parametric Univariate COMP METH v2.00: intro 2/ 35

3 Contact Information Name: Oran Kittithreerapronchai, PhD Office: Room 603, Engineering Building 4 Office Hour: Tuesday 13:00-14:00 Thursday 16:00-17:00 or by appointment oran.k@chula.ac.th Tel: WWW: oran/index.htm COMP METH v2.00: intro 3/ 35

4 Syllabus: Before we start Goals Aware of a general computation method and optimization techniques in IE Optimization (Unconstrainted and Dynamic Programming) Algorithm & Data Mining Analyze IE data for using computation method Requirement Good background in math modeling and optimization Decent programming skills, logic thinking and perseverance COMP METH v2.00: intro 4/ 35

5 Grading Policy Grading Homework & Quiz (20%) Midterm Exam (30%) Term Project: Data Mining (50%) Grading & Scores 85 and above: final grade id definitely A between 50 and 85: A, B +, B, C +,..., D 50 and below: final grade is possibly F COMP METH v2.00: intro 5/ 35

6 Class Rules & Agreements No class attendance Don t interrupt others Focus on your term project Be responsible, especially meeting time and assignment Participate during class; this is Master level course.. Exam is designed to test student basic knowledge Term Project evaluates student performance COMP METH v2.00: intro 6/ 35

7 Code of honors Education with ethic standards and social responsibilities Trust as integral and essential part of learning process Self-discipline necessity Dishonesty hurts the entire community adapted from: Georgia Institute of Technology The Honor Code. Any violation to code of honors will severely punished, especially cheating and. plagiarism COMP METH v2.00: intro 7/ 35

8 Textbook and References Textbook References Chong, E and Zak, S An Introduction to Optimization Wiley-Interscience. Winston, W Operations Research Applications and Algorithms Duxbury Press. Torgo, L Data Mining with R: Learning with Case Studies Chapman & Hall. Zhao, Y R and Data Mining: Examples and Case Studies Elsevier. Mitchell, T.M., 1997 Mitchell Machine Learning McGraw-Hill COMP METH v2.00: intro 8/ 35

9 Class Website Class web site: oran/index.htm What for: teaching materials, assignments, and announcement How to turn-in your assignment Medium: R markdown file or doc file <ID>_hw<#homework>.Rmd (must be self-contain) Where & When: through courseville and before midnight of due date COMP METH v2.00: intro 9/ 35

10 Maximizing Revenue If a company charges a price p USD for a product, then it can sell 3000e p units of the product. What is the revenue function? f(p) = 3000 p e p When does it decreasing/increasing in terms of p p f(p) = 3000 p e p e p = 3000 e p (1 p) increasing: 0 < p < 1 decreasing: 1 < p < Is there any constraint? p 0 Find p that maximize the revenue pf(p) = 0 and solve for p f(p = 1) = source: Winston Chapter 11 Ex03 COMP METH v2.00: intro 10/ 35

11 Multiple ways to solve Trial & Error: time consuming & solution quality always work Graphic: dimensionality provide understanding Gradient: differentiable exact solution Algorithm: combine all above all above COMP METH v2.00: intro 11/ 35

12 Single Facility Location Problem Source. MIT and James Orlin Find the location of single warehouse that minimize the total distances from the following location. (assume all distances are Euclidean.) No. customer location (p i ) # shipments (w i ) 1 A (8,2) 19 2 B (3,10) 7 3 C (8,15) 2 4 D (14,13) 5 COMP METH v2.00: intro 12/ 35

13 Single Facility Location Problem min f(x) = f(x, y) = N w i (x p x i )2 + (y p y i )2 i=1 = 19 (x 8) 2 + (y 2) (x 3) 2 + (y 10) 2 +2 (x 8) 2 + (y 15) (x 14) 2 + (y 13) 2 1 Iter 0 Let d i (x 0, y 0 ) = (x 0 p x i )2 +(y 0 p y i )2 [ N w f(x, y) = i (x p x i ) i=1 d i (x,y), N i=1 w i (y p y i ) d i (x,y) ] T x 0 = (x 0, y 0 ) = (8.25, 10) and f(x 0 ) = Iter 1 x 1 = (, ) and f(x 1 ) = hence, [d 1, d 2, d 3, d 4 ] T = [ ] T Iter 2 x 2 = (, ) and f(x 2 ) = hence, [d 1, d 2, d 3, d 4 ] T = [ ] T Solution x = [8, 2] T and f(x ) = COMP METH v2.00: intro 13/ 35

14 Generalization Let the problem have the general mathematical programming (MP) form (P) minimize f(x) subject to x F. Mathematical programming Yield a solution satisfying constraints feasible solution (F) Solve for good solution minimal value optimal solution (f (x)) COMP METH v2.00: intro 14/ 35

15 Local and Global Minimizer Source. Chong & Zak pp 72. Definition (local minimizer).. Definition (global minimizer). Ạ point x F is a global minimizer of f( ) if f(x) f(x ) for all x F\x COMP METH v2.00: intro 15/ 35

16 Components of Optimization Model Decision Variables: What are we interested in? Objective Function: How do we measure the best solution? Direction: minimize or maximize Constraints: What are the solutions? Parameters: Data needed to describe relationship COMP METH v2.00: intro 16/ 35

17 Optimization Topics Deterministic. Optimization. Stochastic Optimization Unconstraint Non-Linear Univariate Multivariate Constraint Linear Constraint Non-Linear COMP METH v2.00: intro 17/ 35

18 Optimization Cycle Symbolic World Math Model Analysis Comp Result Abstraction. Interpretation Mgt. Situation Intuition Decision Real World COMP METH v2.00: intro 18/ 35

19 What is Data Mining? What: process to discover interesting knowledge from data Tasks: prediction and descriptive (clustering) Why: IT generates many data computer power is cheap business is high competitive Statistic Visualization Machine Learning Data Mining. Inform Sys Computer Alg Database Tech. COMP METH v2.00: intro 19/ 35

20 US Air Passengers AirPassengers Year 1949 Year 1950 Year 1951 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Month COMP METH v2.00: intro 20/ 35

21 Air Passengers De-Seasonality AirPassengers Year 1949 Year 1950 Year 1951 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Month COMP METH v2.00: intro 21/ 35

22 Fraud Transaction Identification A company allows its to set price independently There are many products and saleperson Few percentage can be manually detect fraud Can you build a computer algorithm to identify the fraud? COMP METH v2.00: intro 22/ 35

23 Overwhelming by Big Data COMP METH v2.00: intro 23/ 35

24 Data Mining Task Data Mining Task Predictive Descriptive Classification Regression Clustering Association Time Series Summarization Discovery. COMP METH v2.00: intro 24/ 35

25 Data Mining Cycle Collecting Data Learning Model Building. Obtaining Knowledge Action Business Evaluation COMP METH v2.00: intro 25/ 35

26 Computational Methods in IE Industrial and System Engineering (ISyE) Core: applying systematic thinking to make better decisions Decisions: location, inventory, routing, slotting, planning Role of Data Mining in IE: understanding historical data Role of Optimization in IE: achieving goals (cost, time, risk, regulation, and policies) Target Micro Marketing: customizing promotion for each household Amazon Dynamic Pricing: setting price based on number of views (conceded) COMP METH v2.00: intro 26/ 35

27 What is parametric univariate? Idea: known objective function single DV and no constraint Why: easiest problem Technique: Grid search, First order condition, Newton method Example I min f 1 (x) = x 4 14x x 2 70x, x R Example II min f 2 (x) = 3 x e x2 3 1 x ln(x), x 0 COMP METH v2.00: intro 27/ 35

28 COMP METH v2.00: intro / 35 Grid Search f 1 (x) = x 4 14x x 2 70x, x R

29 First Order Condition f 1 (x) = x 4 14x x 2 70x Then x f 1(x) = 4x 3 42x x 70 and, 2 x 2 f 1(x) = 12x 2 84x Then f 2 (x) = 3 x e x2 3 1 x ln(x) x f 2(x) = 3 e x2 6x 2 e x2 3 1 x 2 + ln(x) x 2 and, 2 x 2 f 2(x) = 6x e x2 12x e x2 + 12x 3 e x x x 3 2 ln(x) x 3 COMP METH v2.00: intro 29/ 35

30 Taylor Series for Univariate.. where, q(x) = f(x k ) + f (x k ) (x x k ) f (x k ) (x x k ) 2 f(x) = Original objective function q(x) = Taylor s approximated objective function First Order Condition x q(x) = f (x k ) + (x x k )f (x k ) = 0 x k+1 = x k f (x k ) f x k COMP METH v2.00: intro 30/ 35

31 What is Newton Method? Source. Chong & Zak pp 106 What: iterative method by estimating a objective function with Taylor s series Advantage: efficient, basic for multiple decision variable Requirement: explicit function, twice continuity COMP METH v2.00: intro 31/ 35

32 Example: Newton Method I Using Newton Method to find solution of this following function, if x 0 = 0.0. f(x) = x 4 14x x 2 70x Source. Chong & Zak pp 93 f 1 (x) = 4x 3 42x x 70 f 1 (x) = 12x 2 84x i x i f(x) f (x i ) f (x i ) x i COMP METH v2.00: intro 32/ 35

33 When is it Time To Quit: Tolerance What do we observe? Solution: virtually unchange, particularly x 3 = and x 4 = Objective Value: virtually unchange, particularly f(x 3 ) = and f(x 4 ) = Slope: virtually zero, particularly particularly f (x 3 ) = and f (x 4 ) = Iterations: take too many iterations Tolerance: ϵ Solution: x k+1 x k < ϵ x Objective Value: f(x k+1 ) f(x k ) < ϵ f(x) Slope: f (x k+1 ) < ϵ f (x) Iterations: t > N COMP METH v2.00: intro 33/ 35

34 Example II: Newton Method Using Newton Method to find solution of this following function using only the first derivative of f 2 ( ) start at x = 1.0. f 2 (x) = 3 x e x2 3 1 x ln(x) f 2 (x) = 3 e x2 6x 2 e x2 3 1 x 2 + ln(x) x 2 f 2 (x) = 6x e x2 12x e x2 + 12x 3 e x x x 3 2 ln(x) x 3 i x i f(x) f (x i ) f (x i ) x i COMP METH v2.00: intro 34/ 35

35 Summary Two Classes of Method Direct search: graphic, grid search Derivative based: FOC, Newton & Quasi-Newton methods Trade-Off Implementation: graphic, grid search Computational load: Convergence speed: FOC, Newton & Quasi-Newton methods COMP METH v2.00: intro 35/ 35

Instructor Dr. Tomislav Pintauer Mellon Hall Office Hours: 1-2 pm on Thursdays and Fridays, and by appointment.

Instructor Dr. Tomislav Pintauer Mellon Hall Office Hours: 1-2 pm on Thursdays and Fridays, and by appointment. Course Description and Objectives The purpose of this course is to introduce students to the basic facts and principles of chemistry. Our approach provides a sound foundation by teaching students that