.. Introduction to Course Oran Kittithreerapronchai 1 1 Department of Industrial Engineering, Chulalongkorn University Bangkok 10330 THAILAND last updated: September 17, 2016 COMP METH v2.00: intro 1/ 35
Outline. 1 Contact Information and Syllabus. 2 Roles and Agreement. 3 Motivation of Optimization. 4 Solving Parametric Univariate COMP METH v2.00: intro 2/ 35
Contact Information Name: Oran Kittithreerapronchai, PhD Office: Room 603, Engineering Building 4 Office Hour: Tuesday 13:00-14:00 Thursday 16:00-17:00 or by appointment Email: oran.k@chula.ac.th Tel: 02-218-6832 WWW: http://www.ie.eng.chula.ac.th/ oran/index.htm http://orankitti.site44.com/index.html COMP METH v2.00: intro 3/ 35
Syllabus: Before we start Goals Aware of a general computation method and optimization techniques in IE Optimization (Unconstrainted and Dynamic Programming) Algorithm & Data Mining Analyze IE data for using computation method Requirement Good background in math modeling and optimization Decent programming skills, logic thinking and perseverance COMP METH v2.00: intro 4/ 35
Grading Policy Grading Homework & Quiz (20%) Midterm Exam (30%) Term Project: Data Mining (50%) Grading & Scores 85 and above: final grade id definitely A between 50 and 85: A, B +, B, C +,..., D 50 and below: final grade is possibly F COMP METH v2.00: intro 5/ 35
Class Rules & Agreements No class attendance Don t interrupt others Focus on your term project Be responsible, especially meeting time and assignment Participate during class; this is Master level course.. Exam is designed to test student basic knowledge Term Project evaluates student performance COMP METH v2.00: intro 6/ 35
Code of honors Education with ethic standards and social responsibilities Trust as integral and essential part of learning process Self-discipline necessity Dishonesty hurts the entire community adapted from: Georgia Institute of Technology The Honor Code. Any violation to code of honors will severely punished, especially cheating and. plagiarism COMP METH v2.00: intro 7/ 35
Textbook and References Textbook References Chong, E and Zak, S. 2001. An Introduction to Optimization Wiley-Interscience. Winston, W.2003. Operations Research Applications and Algorithms Duxbury Press. Torgo, L. 2011 Data Mining with R: Learning with Case Studies Chapman & Hall. Zhao, Y. 2013 R and Data Mining: Examples and Case Studies Elsevier. Mitchell, T.M., 1997 Mitchell Machine Learning McGraw-Hill COMP METH v2.00: intro 8/ 35
Class Website Class web site: https://www.mycourseville.com http://www.ie.eng.chula.ac.th/ oran/index.htm http://orankitti.site44.com/index.html What for: teaching materials, assignments, and announcement How to turn-in your assignment Medium: R markdown file or doc file <ID>_hw<#homework>.Rmd (must be self-contain) Where & When: through courseville and before midnight of due date COMP METH v2.00: intro 9/ 35
Maximizing Revenue If a company charges a price p USD for a product, then it can sell 3000e p units of the product. What is the revenue function? f(p) = 3000 p e p When does it decreasing/increasing in terms of p p f(p) = 3000 p e p + 3000 e p = 3000 e p (1 p) increasing: 0 < p < 1 decreasing: 1 < p < Is there any constraint? p 0 Find p that maximize the revenue pf(p) = 0 and solve for p f(p = 1) = 1103.638 source: Winston. 2003 Chapter 11 Ex03 COMP METH v2.00: intro 10/ 35
Multiple ways to solve Trial & Error: time consuming & solution quality always work Graphic: dimensionality provide understanding Gradient: differentiable exact solution Algorithm: combine all above all above COMP METH v2.00: intro 11/ 35
Single Facility Location Problem Source. MIT and James Orlin. 2003 Find the location of single warehouse that minimize the total distances from the following location. (assume all distances are Euclidean.) No. customer location (p i ) # shipments (w i ) 1 A (8,2) 19 2 B (3,10) 7 3 C (8,15) 2 4 D (14,13) 5 COMP METH v2.00: intro 12/ 35
Single Facility Location Problem min f(x) = f(x, y) = N w i (x p x i )2 + (y p y i )2 i=1 = 19 (x 8) 2 + (y 2) 2 + 7 (x 3) 2 + (y 10) 2 +2 (x 8) 2 + (y 15) 2 + 5 (x 14) 2 + (y 13) 2 1 Iter 0 Let d i (x 0, y 0 ) = (x 0 p x i )2 +(y 0 p y i )2 [ N w f(x, y) = i (x p x i ) i=1 d i (x,y), N i=1 w i (y p y i ) d i (x,y) ] T x 0 = (x 0, y 0 ) = (8.25, 10) and f(x 0 ) = 231.2645 Iter 1 x 1 = (, ) and f(x 1 ) = hence, [d 1, d 2, d 3, d 4 ] T = [ ] T Iter 2 x 2 = (, ) and f(x 2 ) = hence, [d 1, d 2, d 3, d 4 ] T = [ ] T Solution x = [8, 2] T and f(x ) = 154.6877 COMP METH v2.00: intro 13/ 35
Generalization Let the problem have the general mathematical programming (MP) form (P) minimize f(x) subject to x F. Mathematical programming Yield a solution satisfying constraints feasible solution (F) Solve for good solution minimal value optimal solution (f (x)) COMP METH v2.00: intro 14/ 35
Local and Global Minimizer Source. Chong & Zak. 2001 pp 72. Definition (local minimizer).. Definition (global minimizer). Ạ point x F is a global minimizer of f( ) if f(x) f(x ) for all x F\x COMP METH v2.00: intro 15/ 35
Components of Optimization Model Decision Variables: What are we interested in? Objective Function: How do we measure the best solution? Direction: minimize or maximize Constraints: What are the solutions? Parameters: Data needed to describe relationship COMP METH v2.00: intro 16/ 35
Optimization Topics Deterministic. Optimization. Stochastic Optimization Unconstraint Non-Linear Univariate Multivariate Constraint Linear Constraint Non-Linear COMP METH v2.00: intro 17/ 35
Optimization Cycle Symbolic World Math Model Analysis Comp Result Abstraction. Interpretation Mgt. Situation Intuition Decision Real World COMP METH v2.00: intro 18/ 35
What is Data Mining? What: process to discover interesting knowledge from data Tasks: prediction and descriptive (clustering) Why: IT generates many data computer power is cheap business is high competitive Statistic Visualization Machine Learning Data Mining. Inform Sys Computer Alg Database Tech. COMP METH v2.00: intro 19/ 35
US Air Passengers 1949-1951 AirPassengers 60 80 100 120 140 160 180 200 Year 1949 Year 1950 Year 1951 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Month COMP METH v2.00: intro 20/ 35
Air Passengers De-Seasonality AirPassengers 60 80 100 120 140 160 180 200 Year 1949 Year 1950 Year 1951 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Month COMP METH v2.00: intro 21/ 35
Fraud Transaction Identification A company allows its to set price independently There are many products and saleperson Few percentage can be manually detect fraud Can you build a computer algorithm to identify the fraud? COMP METH v2.00: intro 22/ 35
Overwhelming by Big Data COMP METH v2.00: intro 23/ 35
Data Mining Task Data Mining Task Predictive Descriptive Classification Regression Clustering Association Time Series Summarization Discovery. COMP METH v2.00: intro 24/ 35
Data Mining Cycle Collecting Data Learning Model Building. Obtaining Knowledge Action Business Evaluation COMP METH v2.00: intro 25/ 35
Computational Methods in IE Industrial and System Engineering (ISyE) Core: applying systematic thinking to make better decisions Decisions: location, inventory, routing, slotting, planning Role of Data Mining in IE: understanding historical data Role of Optimization in IE: achieving goals (cost, time, risk, regulation, and policies) Target Micro Marketing: customizing promotion for each household Amazon Dynamic Pricing: setting price based on number of views (conceded) COMP METH v2.00: intro 26/ 35
What is parametric univariate? Idea: known objective function single DV and no constraint Why: easiest problem Technique: Grid search, First order condition, Newton method Example I min f 1 (x) = x 4 14x 3 + 60x 2 70x, x R Example II min f 2 (x) = 3 x e x2 3 1 x ln(x), x 0 COMP METH v2.00: intro 27/ 35
COMP METH v2.00: intro 2 1 28/ 35 Grid Search f 1 (x) = x 4 14x 3 + 60x 2 70x, x R
First Order Condition f 1 (x) = x 4 14x 3 + 60x 2 70x Then x f 1(x) = 4x 3 42x 2 + 120x 70 and, 2 x 2 f 1(x) = 12x 2 84x + 120 Then f 2 (x) = 3 x e x2 3 1 x ln(x) x f 2(x) = 3 e x2 6x 2 e x2 3 1 x 2 + ln(x) x 2 and, 2 x 2 f 2(x) = 6x e x2 12x e x2 + 12x 3 e x2 + 6 1 x 3 + 1 x 3 2 ln(x) x 3 COMP METH v2.00: intro 29/ 35
Taylor Series for Univariate.. where, q(x) = f(x k ) + f (x k ) (x x k ) + 1 2 f (x k ) (x x k ) 2 f(x) = Original objective function q(x) = Taylor s approximated objective function First Order Condition x q(x) = f (x k ) + (x x k )f (x k ) = 0 x k+1 = x k f (x k ) f x k COMP METH v2.00: intro 30/ 35
What is Newton Method? Source. Chong & Zak. 2001 pp 106 What: iterative method by estimating a objective function with Taylor s series Advantage: efficient, basic for multiple decision variable Requirement: explicit function, twice continuity COMP METH v2.00: intro 31/ 35
Example: Newton Method I Using Newton Method to find solution of this following function, if x 0 = 0.0. f(x) = x 4 14x 3 + 60x 2 70x Source. Chong & Zak. 2001 pp 93 f 1 (x) = 4x 3 42x 2 + 120x 70 f 1 (x) = 12x 2 84x + 120 i x i f(x) f (x i ) f (x i ) x i+1 0 0.0000 0.0000-70 120. 0 70 120 1 0.5833-23.079-13.5 75.08 0.5833 13.5 75.08 2 0.7630-24.360-1.109 62.888 0.7630 1.109 62.888 3 0.7807-24.369-0.010 61.734 0.7807 0.010 61.734 COMP METH v2.00: intro 32/ 35
When is it Time To Quit: Tolerance What do we observe? Solution: virtually unchange, particularly x 3 = 0.7807 and x 4 = 0.7808 Objective Value: virtually unchange, particularly f(x 3 ) = 24.369 and f(x 4 ) = 24.370 Slope: virtually zero, particularly particularly f (x 3 ) = 1.109 and f (x 4 ) = 0.010 Iterations: take too many iterations Tolerance: ϵ Solution: x k+1 x k < ϵ x Objective Value: f(x k+1 ) f(x k ) < ϵ f(x) Slope: f (x k+1 ) < ϵ f (x) Iterations: t > N COMP METH v2.00: intro 33/ 35
Example II: Newton Method Using Newton Method to find solution of this following function using only the first derivative of f 2 ( ) start at x = 1.0. f 2 (x) = 3 x e x2 3 1 x ln(x) f 2 (x) = 3 e x2 6x 2 e x2 3 1 x 2 + ln(x) x 2 f 2 (x) = 6x e x2 12x e x2 + 12x 3 e x2 + 6 1 x 3 + 1 x 3 2 ln(x) x 3 i x i f(x) f (x i ) f (x i ) x i+1 0 1.0000 1.104-4.104 6.793 1.0000 4.104 6.793 1 1.6041 -.517-1.564 3.069 1.6041 1.564 3.069 2 2.1136 -.989-0.442 1.341 2.1136 0.442 1.341 3 2.4432-1.078-0.137 0.584 2.4432 0.137 0.584 COMP METH v2.00: intro 34/ 35
Summary Two Classes of Method Direct search: graphic, grid search Derivative based: FOC, Newton & Quasi-Newton methods Trade-Off Implementation: graphic, grid search Computational load: Convergence speed: FOC, Newton & Quasi-Newton methods COMP METH v2.00: intro 35/ 35