Linear Classification: Linear Programming
|
|
- Shonda Caroline Craig
- 6 years ago
- Views:
Transcription
1 Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong
2 Recall the definition of linear classification. Definition 1. Let R d denote the d-dimensional space where the domain of each dimension is the set R of real values. Let P be a set of points in R d, each of which is colored either red or blue. The goal of the linear classification problem is to determine whether there is a d-dimensional plane x 1 c 1 + x 2 c x d c d = 0 which separates the red points from the blue points in P. In other words, all the red points must fall on the same side of the plane, while all the blue points must fall on the other side. If the plane exists, then P is said to be linearly separable. Otherwise, P is linearly non-separable.
3 In this lecture, we will give an algorithm that is able to (i) detect whether P is linearly separable, and (ii) if it is, return a separation plane. Our weapon is to convert the problem to another classic problem called linear programming.
4 Definition 2. A half-plane in R d is the set of all points (x 1, x 2,..., x d ) in R d satisfying the following inequality: x 1 c 1 + x 2 c x d c d c d+1 where c 1, c 2,..., c d+1 are real-valued constants. Example 3. y 2 2x + y x 3x 6 (a) A half-plane in R (b) A half-plane in R 2
5 Definition 4 (Linear Programming (LP)). Let S be a set of n half-planes H 1, H 2,..., H n in R d. Let A = H 1 H 2... H n. The goal of the linear programming problem is to decide (i) whether A is empty, and (ii) if A is not empty, return a point in A whose coordinate on the first dimension is the smallest.
6 Example 5 (1d LP). H 1 : x 10, H 2 : x 0, H 3 : x 1, H 4 : x 3, H 5 : x 10 A = [1, 3]; answer: x = 1. H 1 : x 10, H 2 : x 0, H 3 : x 4, H 4 : x 3, H 5 : x 10 A = ; answer: no solution. Example 6 (2d LP). p A A = shadow area; answer: p A = ; answer: no solution
7 The 1d LP problem can be easily solved in O(n) time (recall that n is the number of half-planes). Think How?
8 We now turn our attention to the 2d LP problem. To simplify our discussion, we assume: The planes are in general position. Namely, (i) there do not exist 3 half-planes whose boundary lines cross the same point, and (ii) no boundary line is perpendicular to the x-axis. The optimal solution point is unique. We can ensure this by adding two special planes to S: y x We will assume that these planes are H 1 and H 2 in the discussion below.
9 Now, we give a randomized algorithm to solve the 2d LP problem. Step 1 Randomly permute H 3, H 4,..., H n (we will give a permutation algorithm of running time O(n) in the appendix). Note that the two special planes H 1, H 2 are not permuted. Without loss of generality, let us assume that (H 1,..., H n ) is the sequence of half-planes after the permutation, and l 1,..., l n are their boundary lines, respectively.
10 Step 2 The algorithm will then process the half-planes in the order of H 1, H 2,..., H n. The following invariant will be maintained: after having processed H 1,..., H i, the algorithm will be holding a point p satisfying: If A i = H 1 H 2... H i is not empty, then p is a point with the smallest x-coordinate in A i. Otherwise, p is nil. The point p will become the final answer when the algorithm terminates at i = n. To fulfill the requirement for i = 2, we simply set p to the intersection of l 1 and l 2.
11 Step 3 We process each H i (i 3) by checking whether the current p falls in H i. If so, then the processing on H i is done. Think In this case, p must have the smallest x-coordinate in H 1 H 2... H i. Why?
12 We will first prove a lemma before discussing what to do for the case where p / H i. Lemma 7. If p / H i and A i = H 1... H i is not empty, there must be a point on l i that has the smallest x-coordinate in A i. Proof Suppose that A i is not empty. Let q be a point in A i with the smallest x-coordinate in A i. If q is on l i, then we are done; next, we consider that it is not. Let pq be the line segment connecting p and q. Define A i 1 = H 1... H i 1. A i 1 is a convex region that contains both p and q. It thus follows that A i 1 contains the entire pq.
13 Proof (cont.) Since p and q lie on different sides of l i, we know that pq must intersect l i at a point p. This implies that p falls in all of H 1,..., H i, namely, p A i. p p q l i By definition of p, we know that the x-coordinate of p is less than or equal to that of q. This means that the x-coordinate of p is less than or equal to that of q.
14 Lemma 7 shows that if p / H i, then we can focus on the following problem: Find the point p on l i with the smallest x-coordinate that falls in all of H 1,..., H i 1. For each j [1, i 1], H j intersects l i into a half-line. Hence, there are i 1 half-lines in total. lj l i half-line of h j This is essentially a 1d LP problem defined by all these i 1 half-lines, which we already know can be solved in O(i) time. This completes the algorithm s description.
15 Example 8. p A H5 H1 H4 H3 H1 H3 H2 H2 (a) After permutation (b) After processing H 3 p p H5 H1 H4 H3 H1 H4 H3 H2 H2 (c) After processing H 4 (d) After processing H 5
16 Theorem 9. The algorithm runs in O(n) expected time. Proof As before, let H 1, H 2,..., H n be the sequence of planes after the permutation. Remember that S = {H 3, H 4,..., H n } and H 1, H 2 are the two plans we added manually. The processing of H 1, H 2 clearly takes constant time. For each integer i [3, n], let T i be the time we spend on H i. Denote by T the total running time. Obviously, T = d i=3 T i. Next, we will prove that E[T i ] = O(1) for all i [3, n], which implies that E[T ] = O(n). Fix any i [3, n]. Also, fix a subset Z of S with size Z = n 2 i. Let C(Z) be the event that H 3,..., H i is a permutation of S \ Z. Next, we will prove that E[T i C] = O(1). It will follow immediately from Step 1 (random permutation) that E[T i ] = Z E[T i C(Z)] Pr[C(Z)] = O(1) (think: why?).
17 Proof (cont.) Let A i = H 1... H i. We will discuss only the case where A i is not empty (the other case is left to you). Let p be the point in A i with the smallest x-coordinate. p must be the intersection of the boundary lines of two half-planes, say H j1 and H j2. Observe that: If i j 1 and i j 2, then p was already computed before processing H i. In this case, T i = O(1). Otherwise, the processing of H i needs to solve a 1d LP problem, necessitating T i = O(i). However, due to random permutation, we know that i has at most 2/i probability to come from {j 1, j 2 }. Therefore, E[T i C(Z)] O(1) i 2 i + O(i) 2 i = O(1).
18 Our algorithm can be extended to any dimensionality d. The only change is in Step 3, where we solve a (d 1)-dimensional LP problem if the current p does not fall in H i. As long as d is a constant, the expected running time of the algorithm is still O(n) (the hidden constant is roughly d!).
19 Finally, we mention that LP is often defined in an alternative form: Definition 10 (Linear Programming (LP)). Let S be a set of n half-planes H 1, H 2,..., H n in R d. Let A = H 1 H 2... H n. Also, we are given a linear objective function f (p) that takes as input a point p(x 1,..., x d ) in R d, and returns a real value: f (p) = α 1 x 1 + α 2 x α d x d. The goal of the linear programming problem is to decide whether A is empty. If A is not empty, we also need to return a point p A that minimizes f (p). In the version of Definition 4, f (p) is implicitly defined to be the first coordinate of p. In fact, the above definition, which appears to be more general, is the same as the one in Definition 4. Why?
20 Reduction from Linear Classification to Linear Programming Let us now return to Definition 1. Denote the points in P as p 1, p 2,..., p n, respectively, where n = P. We require that each point p i (x 1,..., x d ) for i [1, n] should satisfy: { x1 c 1 + x 2 c x d c d c d+1 if p i is red x 1 c 1 + x 2 c x d c d c d+1 if p i is blue In this way, we obtain n inequalities with c 1,..., c d+1 being the unknowns. We aim to maximize the value of c d+1. This is an instance of LP. The LP always returns a solution (because at least c 1 = c 2 =... = c d+1 = 0 satisfy all the inequalities). Let c 1,..., c d+1 be the values returned by the LP. We check whether c d+1 = 0. If so, then we declare that P is not linearly separable. Otherwise, x 1 c 1 + x 2 c x d c d = 0 must be a separation plane (the proof is left as an exercise).
21 Appendix: Random Permutation Problem: Let S be an array of n elements. Produce a random permutation of these elements, and store them still in S. Algorithm: for i = 2 to n j = a random number from 1 to i swap S[i] with S[j]
Linear Classification: Linear Programming
Linear Classification: Linear Programming Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong 1 / 21 Y Tao Linear Classification: Linear Programming Recall the definition
More informationc i r i i=1 r 1 = [1, 2] r 2 = [0, 1] r 3 = [3, 4].
Lecture Notes: Rank of a Matrix Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk 1 Linear Independence Definition 1. Let r 1, r 2,..., r m
More informationLinear Classification: Perceptron
Linear Classification: Perceptron Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong 1 / 18 Y Tao Linear Classification: Perceptron In this lecture, we will consider
More informationLecture Notes: Eigenvalues and Eigenvectors. 1 Definitions. 2 Finding All Eigenvalues
Lecture Notes: Eigenvalues and Eigenvectors Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk 1 Definitions Let A be an n n matrix. If there
More informationA = , A 32 = n ( 1) i +j a i j det(a i j). (1) j=1
Lecture Notes: Determinant of a Square Matrix Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk 1 Determinant Definition Let A [a ij ] be an
More information2.2 Some Consequences of the Completeness Axiom
60 CHAPTER 2. IMPORTANT PROPERTIES OF R 2.2 Some Consequences of the Completeness Axiom In this section, we use the fact that R is complete to establish some important results. First, we will prove that
More informationj=1 u 1jv 1j. 1/ 2 Lemma 1. An orthogonal set of vectors must be linearly independent.
Lecture Notes: Orthogonal and Symmetric Matrices Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk Orthogonal Matrix Definition. Let u = [u
More informationConsequences of the Completeness Property
Consequences of the Completeness Property Philippe B. Laval KSU Today Philippe B. Laval (KSU) Consequences of the Completeness Property Today 1 / 10 Introduction In this section, we use the fact that R
More information1 Review Session. 1.1 Lecture 2
1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions
More informationNATIONAL UNIVERSITY OF SINGAPORE Department of Mathematics MA4247 Complex Analysis II Lecture Notes Part II
NATIONAL UNIVERSITY OF SINGAPORE Department of Mathematics MA4247 Complex Analysis II Lecture Notes Part II Chapter 2 Further properties of analytic functions 21 Local/Global behavior of analytic functions;
More informationChapter 2: Linear Programming Basics. (Bertsimas & Tsitsiklis, Chapter 1)
Chapter 2: Linear Programming Basics (Bertsimas & Tsitsiklis, Chapter 1) 33 Example of a Linear Program Remarks. minimize 2x 1 x 2 + 4x 3 subject to x 1 + x 2 + x 4 2 3x 2 x 3 = 5 x 3 + x 4 3 x 1 0 x 3
More informationCS5314 Randomized Algorithms. Lecture 18: Probabilistic Method (De-randomization, Sample-and-Modify)
CS5314 Randomized Algorithms Lecture 18: Probabilistic Method (De-randomization, Sample-and-Modify) 1 Introduce two topics: De-randomize by conditional expectation provides a deterministic way to construct
More information. The following is a 3 3 orthogonal matrix: 2/3 1/3 2/3 2/3 2/3 1/3 1/3 2/3 2/3
Lecture Notes: Orthogonal and Symmetric Matrices Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk Orthogonal Matrix Definition. An n n matrix
More information3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions
A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence
More informationToday s exercises. 5.17: Football Pools. 5.18: Cells of Line and Hyperplane Arrangements. Inclass: PPZ on the formula F
Exercise Session 9 03.05.2016 slide 1 Today s exercises 5.17: Football Pools 5.18: Cells of Line and Hyperplane Arrangements Inclass: PPZ on the formula F 6.1: Harmonic vs. Geometric Mean 6.2: Many j-isolated
More informationCS 374: Algorithms & Models of Computation, Spring 2017 Greedy Algorithms Lecture 19 April 4, 2017 Chandra Chekuri (UIUC) CS374 1 Spring / 1
CS 374: Algorithms & Models of Computation, Spring 2017 Greedy Algorithms Lecture 19 April 4, 2017 Chandra Chekuri (UIUC) CS374 1 Spring 2017 1 / 1 Part I Greedy Algorithms: Tools and Techniques Chandra
More informationCSE 421 Greedy Algorithms / Interval Scheduling
CSE 421 Greedy Algorithms / Interval Scheduling Yin Tat Lee 1 Interval Scheduling Job j starts at s(j) and finishes at f(j). Two jobs compatible if they don t overlap. Goal: find maximum subset of mutually
More informationMath 105A HW 1 Solutions
Sect. 1.1.3: # 2, 3 (Page 7-8 Math 105A HW 1 Solutions 2(a ( Statement: Each positive integers has a unique prime factorization. n N: n = 1 or ( R N, p 1,..., p R P such that n = p 1 p R and ( n, R, S
More informationCharacterisation of Accumulation Points. Convergence in Metric Spaces. Characterisation of Closed Sets. Characterisation of Closed Sets
Convergence in Metric Spaces Functional Analysis Lecture 3: Convergence and Continuity in Metric Spaces Bengt Ove Turesson September 4, 2016 Suppose that (X, d) is a metric space. A sequence (x n ) X is
More information5 ProbabilisticAnalysisandRandomized Algorithms
5 ProbabilisticAnalysisandRandomized Algorithms This chapter introduces probabilistic analysis and randomized algorithms. If you are unfamiliar with the basics of probability theory, you should read Appendix
More informationCosets and Lagrange s theorem
Cosets and Lagrange s theorem These are notes on cosets and Lagrange s theorem some of which may already have been lecturer. There are some questions for you included in the text. You should write the
More informationMath 110 (S & E) Textbook: Calculus Early Transcendentals by James Stewart, 7 th Edition
Math 110 (S & E) Textbook: Calculus Early Transcendentals by James Stewart, 7 th Edition 1 Appendix A : Numbers, Inequalities, and Absolute Values Sets A set is a collection of objects with an important
More informationChapter 4. Measure Theory. 1. Measure Spaces
Chapter 4. Measure Theory 1. Measure Spaces Let X be a nonempty set. A collection S of subsets of X is said to be an algebra on X if S has the following properties: 1. X S; 2. if A S, then A c S; 3. if
More informationLinear Algebra Review: Linear Independence. IE418 Integer Programming. Linear Algebra Review: Subspaces. Linear Algebra Review: Affine Independence
Linear Algebra Review: Linear Independence IE418: Integer Programming Department of Industrial and Systems Engineering Lehigh University 21st March 2005 A finite collection of vectors x 1,..., x k R n
More informationLecture 2: A Las Vegas Algorithm for finding the closest pair of points in the plane
Randomized Algorithms Lecture 2: A Las Vegas Algorithm for finding the closest pair of points in the plane Sotiris Nikoletseas Professor CEID - ETY Course 2017-2018 Sotiris Nikoletseas, Professor Randomized
More informationThe Tychonoff Theorem
Requirement for Department of Mathematics Lovely Professional University Punjab, India February 7, 2014 Outline Ordered Sets Requirement for 1 Ordered Sets Outline Ordered Sets Requirement for 1 Ordered
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 7: Duality and applications Prof. John Gunnar Carlsson September 29, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 29, 2010 1
More informationADVANCED CALCULUS - MTH433 LECTURE 4 - FINITE AND INFINITE SETS
ADVANCED CALCULUS - MTH433 LECTURE 4 - FINITE AND INFINITE SETS 1. Cardinal number of a set The cardinal number (or simply cardinal) of a set is a generalization of the concept of the number of elements
More informationAlgorithms, Probability, and Computing Final Exam HS14
Institute of Theoretical Computer Science Thomas Holenstein, Emo Welzl, Peter Widmayer Algorithms, Probability, and Computing Final Exam HS14 Candidate First name:.........................................................
More informationIntroduction to Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras
Introduction to Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Module - 03 Simplex Algorithm Lecture 15 Infeasibility In this class, we
More informationHomework #2 Solutions Due: September 5, for all n N n 3 = n2 (n + 1) 2 4
Do the following exercises from the text: Chapter (Section 3):, 1, 17(a)-(b), 3 Prove that 1 3 + 3 + + n 3 n (n + 1) for all n N Proof The proof is by induction on n For n N, let S(n) be the statement
More informationNearest Neighbor Search with Keywords: Compression
Nearest Neighbor Search with Keywords: Compression Yufei Tao KAIST June 3, 2013 In this lecture, we will continue our discussion on: Problem (Nearest Neighbor Search with Keywords) Let P be a set of points
More informationA NICE PROOF OF FARKAS LEMMA
A NICE PROOF OF FARKAS LEMMA DANIEL VICTOR TAUSK Abstract. The goal of this short note is to present a nice proof of Farkas Lemma which states that if C is the convex cone spanned by a finite set and if
More informationMA554 Assessment 1 Cosets and Lagrange s theorem
MA554 Assessment 1 Cosets and Lagrange s theorem These are notes on cosets and Lagrange s theorem; they go over some material from the lectures again, and they have some new material it is all examinable,
More information1 The linear algebra of linear programs (March 15 and 22, 2015)
1 The linear algebra of linear programs (March 15 and 22, 2015) Many optimization problems can be formulated as linear programs. The main features of a linear program are the following: Variables are real
More information1 The Well Ordering Principle, Induction, and Equivalence Relations
1 The Well Ordering Principle, Induction, and Equivalence Relations The set of natural numbers is the set N = f1; 2; 3; : : :g. (Some authors also include the number 0 in the natural numbers, but number
More informationCS261: Problem Set #3
CS261: Problem Set #3 Due by 11:59 PM on Tuesday, February 23, 2016 Instructions: (1) Form a group of 1-3 students. You should turn in only one write-up for your entire group. (2) Submission instructions:
More information3 Measurable Functions
3 Measurable Functions Notation A pair (X, F) where F is a σ-field of subsets of X is a measurable space. If µ is a measure on F then (X, F, µ) is a measure space. If µ(x) < then (X, F, µ) is a probability
More informationChapter 12: Differentiation. SSMth2: Basic Calculus Science and Technology, Engineering and Mathematics (STEM) Strands Mr. Migo M.
Chapter 12: Differentiation SSMth2: Basic Calculus Science and Technology, Engineering and Mathematics (STEM) Strands Mr. Migo M. Mendoza Chapter 12: Differentiation Lecture 12.1: The Derivative Lecture
More informationCOMBINATORIAL GROUP THEORY NOTES
COMBINATORIAL GROUP THEORY NOTES These are being written as a companion to Chapter 1 of Hatcher. The aim is to give a description of some of the group theory required to work with the fundamental groups
More informationLecture Notes: Solving Linear Systems with Gauss Elimination
Lecture Notes: Solving Linear Systems with Gauss Elimination Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk 1 Echelon Form and Elementary
More informationAugust 23, 2017 Let us measure everything that is measurable, and make measurable everything that is not yet so. Galileo Galilei. 1.
August 23, 2017 Let us measure everything that is measurable, and make measurable everything that is not yet so. Galileo Galilei 1. Vector spaces 1.1. Notations. x S denotes the fact that the element x
More informationLecture Notes: Matrix Inverse. 1 Inverse Definition. 2 Inverse Existence and Uniqueness
Lecture Notes: Matrix Inverse Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk Inverse Definition We use I to represent identity matrices,
More information1 Primals and Duals: Zero Sum Games
CS 124 Section #11 Zero Sum Games; NP Completeness 4/15/17 1 Primals and Duals: Zero Sum Games We can represent various situations of conflict in life in terms of matrix games. For example, the game shown
More informationChapter. Triangles. Copyright Cengage Learning. All rights reserved.
Chapter 3 Triangles Copyright Cengage Learning. All rights reserved. 3.5 Inequalities in a Triangle Copyright Cengage Learning. All rights reserved. Inequalities in a Triangle Important inequality relationships
More informationConvex Sets with Applications to Economics
Convex Sets with Applications to Economics Debasis Mishra March 10, 2010 1 Convex Sets A set C R n is called convex if for all x, y C, we have λx+(1 λ)y C for all λ [0, 1]. The definition says that for
More informationGleason s theorem and unentangled orthonormal bases
Gleason s theorem and unentangled orthonormal bases Nolan R. Wallach [5/14]May, 2014 Nolan R. Wallach () Gleason s theorem and unentangled orthonormal bases [5/14]May, 2014 1 / 19 Nolan R. Wallach () Gleason
More informationHow can we find the distance between a point and a plane in R 3? Between two lines in R 3? Between two planes? Between a plane and a line?
Overview Yesterday we introduced equations to describe lines and planes in R 3 : r = r 0 + tv The vector equation for a line describes arbitrary points r in terms of a specific point r 0 and the direction
More information1 Matroid intersection
CS 369P: Polyhedral techniques in combinatorial optimization Instructor: Jan Vondrák Lecture date: October 21st, 2010 Scribe: Bernd Bandemer 1 Matroid intersection Given two matroids M 1 = (E, I 1 ) and
More informationCOMP3121/9101/3821/9801 Lecture Notes. Linear Programming
COMP3121/9101/3821/9801 Lecture Notes Linear Programming LiC: Aleks Ignjatovic THE UNIVERSITY OF NEW SOUTH WALES School of Computer Science and Engineering The University of New South Wales Sydney 2052,
More informationNotes for Functional Analysis
Notes for Functional Analysis Wang Zuoqin (typed by Xiyu Zhai) November 6, 2015 1 Lecture 18 1.1 The convex hull Let X be any vector space, and E X a subset. Definition 1.1. The convex hull of E is the
More informationStudying Rudin s Principles of Mathematical Analysis Through Questions. August 4, 2008
Studying Rudin s Principles of Mathematical Analysis Through Questions Mesut B. Çakır c August 4, 2008 ii Contents 1 The Real and Complex Number Systems 3 1.1 Introduction............................................
More informationST5215: Advanced Statistical Theory
Department of Statistics & Applied Probability Wednesday, October 5, 2011 Lecture 13: Basic elements and notions in decision theory Basic elements X : a sample from a population P P Decision: an action
More informationMATH1050 Greatest/least element, upper/lower bound
MATH1050 Greatest/ element, upper/lower bound 1 Definition Let S be a subset of R x λ (a) Let λ S λ is said to be a element of S if, for any x S, x λ (b) S is said to have a element if there exists some
More informationIntroduction to Real Analysis Alternative Chapter 1
Christopher Heil Introduction to Real Analysis Alternative Chapter 1 A Primer on Norms and Banach Spaces Last Updated: March 10, 2018 c 2018 by Christopher Heil Chapter 1 A Primer on Norms and Banach Spaces
More informationDOMINO TILINGS INVARIANT GIBBS MEASURES
DOMINO TILINGS and their INVARIANT GIBBS MEASURES Scott Sheffield 1 References on arxiv.org 1. Random Surfaces, to appear in Asterisque. 2. Dimers and amoebae, joint with Kenyon and Okounkov, to appear
More informationLecture 2. MATH3220 Operations Research and Logistics Jan. 8, Pan Li The Chinese University of Hong Kong. Integer Programming Formulations
Lecture 2 MATH3220 Operations Research and Logistics Jan. 8, 2015 Pan Li The Chinese University of Hong Kong 2.1 Agenda 1 2 3 2.2 : a linear program plus the additional constraints that some or all of
More informationMath 530 Lecture Notes. Xi Chen
Math 530 Lecture Notes Xi Chen 632 Central Academic Building, University of Alberta, Edmonton, Alberta T6G 2G1, CANADA E-mail address: xichen@math.ualberta.ca 1991 Mathematics Subject Classification. Primary
More informationIsomorphisms between pattern classes
Journal of Combinatorics olume 0, Number 0, 1 8, 0000 Isomorphisms between pattern classes M. H. Albert, M. D. Atkinson and Anders Claesson Isomorphisms φ : A B between pattern classes are considered.
More informationHidden Markov Models Part 1: Introduction
Hidden Markov Models Part 1: Introduction CSE 6363 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1 Modeling Sequential Data Suppose that
More informationComputational Geometry Lecture Linear Programming
Computational Geometry Lecture Linear Programming INSTITUTE FOR THEORETICAL INFORMATICS FACULTY OF INFORMATICS Tamara Mchedlidze Darren Strash 09.11.2015 1 Profit optimization You are the boss of a company,
More informationx 1 + x 2 2 x 1 x 2 1 x 2 2 min 3x 1 + 2x 2
Lecture 1 LPs: Algebraic View 1.1 Introduction to Linear Programming Linear programs began to get a lot of attention in 1940 s, when people were interested in minimizing costs of various systems while
More informationVectors. Section 3: Using the vector product
Vectors Section 3: Using the vector product Notes and Examples These notes contain subsections on Using the vector product in finding the equation of a plane The intersection of two planes The distance
More informationSorting. Chapter 11. CSE 2011 Prof. J. Elder Last Updated: :11 AM
Sorting Chapter 11-1 - Sorting Ø We have seen the advantage of sorted data representations for a number of applications q Sparse vectors q Maps q Dictionaries Ø Here we consider the problem of how to efficiently
More informationDefinition 2.3. We define addition and multiplication of matrices as follows.
14 Chapter 2 Matrices In this chapter, we review matrix algebra from Linear Algebra I, consider row and column operations on matrices, and define the rank of a matrix. Along the way prove that the row
More information18.175: Lecture 2 Extension theorems, random variables, distributions
18.175: Lecture 2 Extension theorems, random variables, distributions Scott Sheffield MIT Outline Extension theorems Characterizing measures on R d Random variables Outline Extension theorems Characterizing
More informationVector Spaces and Dimension. Subspaces of. R n. addition and scalar mutiplication. That is, if u, v in V and alpha in R then ( u + v) Exercise: x
Vector Spaces and Dimension Subspaces of Definition: A non-empty subset V is a subspace of if V is closed under addition and scalar mutiplication. That is, if u, v in V and alpha in R then ( u + v) V and
More informationLecture 2: Divide and conquer and Dynamic programming
Chapter 2 Lecture 2: Divide and conquer and Dynamic programming 2.1 Divide and Conquer Idea: - divide the problem into subproblems in linear time - solve subproblems recursively - combine the results in
More informationRelations, Functions, and Sequences
MCS-236: Graph Theory Handout #A3 San Skulrattanakulchai Gustavus Adolphus College Sep 13, 2010 Relations, Functions, and Sequences Relations An ordered pair can be constructed from any two mathematical
More informationEQUIVALENCE RELATIONS (NOTES FOR STUDENTS) 1. RELATIONS
EQUIVALENCE RELATIONS (NOTES FOR STUDENTS) LIOR SILBERMAN Version 1.0 compiled September 9, 2015. 1.1. List of examples. 1. RELATIONS Equality of real numbers: for some x,y R we have x = y. For other pairs
More informationChapter 0 Preliminaries
Chapter 0 Preliminaries MA1101 Mathematics 1A Semester I Year 2017/2018 FTMD & FTI International Class Odd NIM (K-46) Lecturer: Dr. Rinovia Simanjuntak 0.1 Real Numbers and Logic Real Numbers Repeating
More information5 Group theory. 5.1 Binary operations
5 Group theory This section is an introduction to abstract algebra. This is a very useful and important subject for those of you who will continue to study pure mathematics. 5.1 Binary operations 5.1.1
More information1 The Erdős Ko Rado Theorem
1 The Erdős Ko Rado Theorem A family of subsets of a set is intersecting if any two elements of the family have at least one element in common It is easy to find small intersecting families; the basic
More informationHomework 3. Convex Optimization /36-725
Homework 3 Convex Optimization 10-725/36-725 Due Friday October 14 at 5:30pm submitted to Christoph Dann in Gates 8013 (Remember to a submit separate writeup for each problem, with your name at the top)
More information1: Introduction to Lattices
CSE 206A: Lattice Algorithms and Applications Winter 2012 Instructor: Daniele Micciancio 1: Introduction to Lattices UCSD CSE Lattices are regular arrangements of points in Euclidean space. The simplest
More informationLinear Programming. Chapter Introduction
Chapter 3 Linear Programming Linear programs (LP) play an important role in the theory and practice of optimization problems. Many COPs can directly be formulated as LPs. Furthermore, LPs are invaluable
More informationVECTORS AND THE GEOMETRY OF SPACE
VECTORS AND THE GEOMETRY OF SPACE VECTORS AND THE GEOMETRY OF SPACE A line in the xy-plane is determined when a point on the line and the direction of the line (its slope or angle of inclination) are given.
More informationInteger Programming ISE 418. Lecture 8. Dr. Ted Ralphs
Integer Programming ISE 418 Lecture 8 Dr. Ted Ralphs ISE 418 Lecture 8 1 Reading for This Lecture Wolsey Chapter 2 Nemhauser and Wolsey Sections II.3.1, II.3.6, II.4.1, II.4.2, II.5.4 Duality for Mixed-Integer
More informationRandomized Sorting Algorithms Quick sort can be converted to a randomized algorithm by picking the pivot element randomly. In this case we can show th
CSE 3500 Algorithms and Complexity Fall 2016 Lecture 10: September 29, 2016 Quick sort: Average Run Time In the last lecture we started analyzing the expected run time of quick sort. Let X = k 1, k 2,...,
More informationSkylines. Yufei Tao. ITEE University of Queensland. INFS4205/7205, Uni of Queensland
Yufei Tao ITEE University of Queensland Today we will discuss problems closely related to the topic of multi-criteria optimization, where one aims to identify objects that strike a good balance often optimal
More informationLECTURE 15: COMPLETENESS AND CONVEXITY
LECTURE 15: COMPLETENESS AND CONVEXITY 1. The Hopf-Rinow Theorem Recall that a Riemannian manifold (M, g) is called geodesically complete if the maximal defining interval of any geodesic is R. On the other
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.433: Combinatorial Optimization Michel X. Goemans February 28th, 2013 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationMATH31011/MATH41011/MATH61011: FOURIER ANALYSIS AND LEBESGUE INTEGRATION. Chapter 2: Countability and Cantor Sets
MATH31011/MATH41011/MATH61011: FOURIER ANALYSIS AND LEBESGUE INTEGRATION Chapter 2: Countability and Cantor Sets Countable and Uncountable Sets The concept of countability will be important in this course
More informationBaltic Way 2003 Riga, November 2, 2003
altic Way 2003 Riga, November 2, 2003 Problems and solutions. Let Q + be the set of positive rational numbers. Find all functions f : Q + Q + which for all x Q + fulfil () f ( x ) = f (x) (2) ( + x ) f
More informationComputational Integer Programming Universidad de los Andes. Lecture 1. Dr. Ted Ralphs
Computational Integer Programming Universidad de los Andes Lecture 1 Dr. Ted Ralphs MIP Lecture 1 1 Quick Introduction Bio Course web site Course structure http://coral.ie.lehigh.edu/ ted/teaching/mip
More informationLinear Programming and its Extensions Prof. Prabha Shrama Department of Mathematics and Statistics Indian Institute of Technology, Kanpur
Linear Programming and its Extensions Prof. Prabha Shrama Department of Mathematics and Statistics Indian Institute of Technology, Kanpur Lecture No. # 03 Moving from one basic feasible solution to another,
More informationCourse 212: Academic Year Section 1: Metric Spaces
Course 212: Academic Year 1991-2 Section 1: Metric Spaces D. R. Wilkins Contents 1 Metric Spaces 3 1.1 Distance Functions and Metric Spaces............. 3 1.2 Convergence and Continuity in Metric Spaces.........
More informationGroups Subgroups Normal subgroups Quotient groups Homomorphisms Cyclic groups Permutation groups Cayley s theorem Class equations Sylow theorems
Group Theory Groups Subgroups Normal subgroups Quotient groups Homomorphisms Cyclic groups Permutation groups Cayley s theorem Class equations Sylow theorems Groups Definition : A non-empty set ( G,*)
More informationCS 125 Section #12 (More) Probability and Randomized Algorithms 11/24/14. For random numbers X which only take on nonnegative integer values, E(X) =
CS 125 Section #12 (More) Probability and Randomized Algorithms 11/24/14 1 Probability First, recall a couple useful facts from last time about probability: Linearity of expectation: E(aX + by ) = ae(x)
More informationSequential programs. Uri Abraham. March 9, 2014
Sequential programs Uri Abraham March 9, 2014 Abstract In this lecture we deal with executions by a single processor, and explain some basic notions which are important for concurrent systems as well.
More informationMathematics for Economists
Mathematics for Economists Victor Filipe Sao Paulo School of Economics FGV Metric Spaces: Basic Definitions Victor Filipe (EESP/FGV) Mathematics for Economists Jan.-Feb. 2017 1 / 34 Definitions and Examples
More informationPart V. Chapter 19. Congruence of integers
Part V. Chapter 19. Congruence of integers Congruence modulo m Let m be a positive integer. Definition. Integers a and b are congruent modulo m if and only if a b is divisible by m. For example, 1. 277
More information2. Introduction to commutative rings (continued)
2. Introduction to commutative rings (continued) 2.1. New examples of commutative rings. Recall that in the first lecture we defined the notions of commutative rings and field and gave some examples of
More informationVector equations of lines in the plane and 3-space (uses vector addition & scalar multiplication).
Boise State Math 275 (Ultman) Worksheet 1.6: Lines and Planes From the Toolbox (what you need from previous classes) Plotting points, sketching vectors. Be able to find the component form a vector given
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 3: Linear Programming, Continued Prof. John Gunnar Carlsson September 15, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, 2010
More informationLecture 10: Linear programming duality and sensitivity 0-0
Lecture 10: Linear programming duality and sensitivity 0-0 The canonical primal dual pair 1 A R m n, b R m, and c R n maximize z = c T x (1) subject to Ax b, x 0 n and minimize w = b T y (2) subject to
More informationTopics in Theoretical Computer Science April 08, Lecture 8
Topics in Theoretical Computer Science April 08, 204 Lecture 8 Lecturer: Ola Svensson Scribes: David Leydier and Samuel Grütter Introduction In this lecture we will introduce Linear Programming. It was
More informationVC Dimension Review. The purpose of this document is to review VC dimension and PAC learning for infinite hypothesis spaces.
VC Dimension Review The purpose of this document is to review VC dimension and PAC learning for infinite hypothesis spaces. Previously, in discussing PAC learning, we were trying to answer questions about
More informationPacking Congruent Bricks into a Cube
Journal for Geometry and Graphics Volume 5 (2001), No. 1, 1 11. Packing Congruent Bricks into a Cube Ákos G. Horváth, István Prok Department of Geometry, Budapest University of Technology and Economics
More information