Paris. Optimization. (Paris 1 Panthéon-Sorbonne and PSE) Paris, 2017.
Lecture 1: About Optimization A For QEM-MMEF, the course (3H each week) and tutorial (4 hours each week) from now to october 22. Exam for this course at end october. Probably one more grade (exam end september). Grade take into account attendency, the two grades of the exams, and participation during the tutorial. For IMAEF, the course and tutorial from now to october 22. Exam for this course=one at end october (with QEM-MMEF) and one in January (with MAEF). Grade take into account attendency, the exams, and participation during the tutorial. For MAEF, the course from now to october 22, but the tutorial from now to december (because 2H each week only) Exam for this course at the end. grade take into account attendency, the final grade of the exam, participation during the tutorial, and some exams during the tutorial.
Lecture 1: About Optimization A I will send you the week before the course+tutorial. I ask you to read the course in details and actively, so that you come during the course with questions: I will take into account that you have already red the course. Also, you have to prepair the exercises. For people who want to work the background material for this course, see: ESSENTIAL MATHEMATICS FOR Economic Analysis, FOURTH EDITION, from Knut Sydsæter and Peter Hammond with Arne Strøm, in particular chapters 1 to 7, and 11-12. The same book treat a large part of our course, but not everything, and with some notations and methods sometimes different, so only my course and exercises are a reference for the exam! it is sufficient to pass the exam!
To wake up There is an ant on a cube placed at one corner S and you need to find the shortest path to the diagonally opposite corner Q (the ant can not fly and cannot enter into the cube).
A first possible motivation of Optimization (beyond the level of this course!): Shape Optimization Let γ be a given curve in R 3. What is the surface whose boundary is γ and which has the minimal area? (this is equivalent to minimize the energy due to superficial tension)?
Lecture 1: Chapter 1: Optimization (vocabulary) Just recall that f : E R is bounded below on C E if there exists m R (called a minorant of f on C) such that x C, m f (x). If this condition is true, the infimum of f on C, denoted inf x C f (x), is a real and is the greatest minorant of f on C (it means that for every m minorant of f on C, we have m inf x C f (x).) If there exists x C such that f ( x) = inf x X f (x), we note min x X f (x) instead of inf x X f (x), and we say that the infimum is reached. In practice, α = inf x C f (x) is the unique real that satisfies: (i) x C, α f (x). (ii) (x n ) n IN sequence of C such that lim f (x n) = α. n +
Chapter 1: Optimization (vocabulary) Just recall that f : E R is bounded above on C E if there exists M R (called a majorant of f on C) if x C, M f (x). If this condition is true, the supremum of f on C, denoted sup x C f (x), is a real and is the lowest majorant of f on C (it means that for every M majorant of f on C, we have M sup x C f (x).) If there exists x C such that f ( x) = sup x X f (x), we note max x X f (x) instead of sup x X f (x), and we say that the supremum is reached. In practice, β = sup x C f (x) is the only real that satisfies: (i) x C, β f (x). (ii) (x n ) n IN sequence of C such that lim f (x n) = β. n +
Chapter 1: Optimization (vocabulary) Let f : E R, and denote D(f ) the domain of f. Let C D(f ). Consider the maximization and minimization problems: (P) max x C f (x). (Q) min x C f (x). f is the objective function, C the set of feasible points (or set of constraints). The value of (P) (resp. Q) is Val(P) = sup f (x) x C Val(Q) = inf x C f (x). By convention, we will write Val(P) = + when f is not bounded above on C, and Val(Q) = when f is not bounded below on C.
Chapter 1: Optimization (vocabulary) Let f : E R, and denote D(f ) the domain of f. Let C D(f ). x E is a solution of (P) max x C f (x) (sometimes we will say also global solution of (P) on C) if val(p) = f ( x) x E is a solution of (Q) min x C f (x) (sometimes we will say also global solution of (Q) on C) if val(q) = f ( x)
Chapter 1: Optimization (vocabulary) To define local solution, we now assume D(f ) IR n for some integer n, endowed with the euclidean distance. x E is a local solution of (P) max x C f (x) if ε > 0, x B( x, ε), f (x) f ( x) x E is a local solution of (Q) min x C f (x) if ε > 0, x B( x, ε), f (x) f ( x)
Chapter 1: Optimization (vocabulary) Let (P) max f (x). x C A maximizing sequence for (P) is any sequence (x n ) of C such that the sequence (f (x n )) converges to Val(P). Let (Q) min f (x). x C A minimizing sequence for (P) is any sequence (x n ) of C such that the sequence (f (x n )) converges to Val(Q). Theorem There always exists a maximizing sequence or a minimizing sequence. Method! To prove that an optimization problem has a solution, a general method is to consider a minimizing or maximizing sequence, and to try to prove that it converges to an optimum (when possible!)
Chapter 1: Existence of solutions of optimization problem when f goes from R to R For C R, consider the problem (P) min x C f (x) Method! A method to solve (P), if possible, simply study the variations of f on C, which permits to get local maxima or local minima, and compare with values (or limits) of f at boundary points of C. Example: (P) min x + 1 x R x Method! For some problems with several variables, the constraint permits to eliminate variables and to get only one variable. Example : (P) min (x,y) [0,1] [0,1]:x+y=1 x2 + y 2
Chapter 1: Exercise Consider (P) max x C e x x. Find value, solution, maximizing sequence when C = [1, 2], when C = [0, 10], when C = [0, + [. Consider e x (Q) min x C x. Find value, solution, local solution, when C =], 0[ ]0, + [.
Lecture 1: Chapter 2: Existence of solutions of optimization problem In this chapter, we try to see if we can define simple criteria on the functino, the domain,...to be sure there exists at least one solution of some optimization problem.
Lecture 1: Chapter 2: Existence of solutions of optimization problem Section 1: Reminders Reminder 1 We recall that a metric space (K, d) is said to be closed if for every sequence of K that converges, its limit has to belong to K. In practice, we often use the following criterium to prove closeness: Method: Practical Criterium to prove closeness If C = f 1 (I) := {x R n : f (x) I} where f is a continuous mapping from R n to R and I a closed subset R, then C is a closed subset of R n. Reminder 2 We recall that a subsequence of (x n) is a sequence (x φ(n) ) where φ is a strictly increasing mapping from N to itself. Reminder 3 A compact subset K of R n is a closed and bounded subset of R n. Bolzano-Weirstrass property gives that "K R n is compact if and only if "Every sequence (x n) of K admits a subsequence (x φ(n) ) which converges in K."
Chapter 2: Existence of solutions of optimization problem Section 2: A first criterium for the existence of a solution of max or min problems in finite dimension Theorem 1 (Weierstrass s Theorem) Let f : C R continuous, where C is a compact subset of R n. Then has at least one solution, and (P) min x C f (x) has at least one solution. Proof. (Q) max x C f (x)
Little test (no grade) Question 1 For which x real is the function f (x) = 1 + x 2 well defined?
Little test (no grade) Question 2 Find the minimum of the function x 2 + x 3.
Little test (no grade) Question 3 What is the limit of x2 +x+1 x+10 when x tends to +?
Little test (no grade) Question 4 Is the function x = x 2 differentiable at every real? is it continuous at every real?
Little test (no grade) Question 5 Compute the sum of all integers beween 1 and 100.
Little test (no grade) Question 6 factorize x 2 2.
Little test (no grade) Question 7 Prove that sin(x) x for every x 0.
Little test (no grade) Question 8 Let u n a sequence of reals which tends to + when n tends to +. Is (u n ) an increasing sequence?
Little test (no grade) Question 9 Let (u n ) a sequence converging to 1. Does there exists some integer n such that u n = 1?
Little test (no grade) Question 9 Let (u n ) a sequence converging to 1. Does there exists some integer n such that u n = 1?