Optimization: Insights and Applications. Jan Brinkhuis Vladimir Tikhomirov PRINCETON UNIVERSITY PRESS PRINCETON AND OXFORD

Transcription

1 Optimization: Insights and Applications Jan Brinkhuis Vladimir Tikhomirov PRINCETON UNIVERSITY PRESS PRINCETON AND OXFORD

2 Contents Preface 0.1 Optimization: insights and applications xiii 0.2 Lunch, dinner, and dessert xiv 0.3 For whom is this book meant? xvi 0.4 What is in this book? xviii 0.5 Special features xix Necessary Conditions: What Is the Point? 1 xi Chapter 1. Fermat: One Variable without Constraints Summary Introduction The derivative for one variable Main result: Fermat theorem for one variable Applications to concrete problems Discussion and comments Exercises 59 Chapter 2. Fermat: Two or More Variables without Constraints Summary \ Introduction The derivative for two or more variables Main result: Fermat theorem for two or more variables Applications to concrete problems Discussion and comments Exercises 128 Chapter 3. Lagrange: Equality Constraints Summary Introduction Main result: Lagrange multiplier rule Applications to concrete problems Proof of the Lagrange multiplier rule Discussion and comments Exercises 190

3 viii Chapter 4. Inequality Constraints and Convexity Summary Introduction Main result: Karush-Kuhn-Tucker theorem Applications to concrete problems Proof of the Karush-Kuhn-Tucker theorem Discussion and comments Exercises 250 Chapter 5. Second Order Conditions Summary Introduction Main result: second order conditions Applications to concrete problems Discussion and comments Exercises 272 Chapter 6. Basic Algorithms Summary Introduction Nonlinear optimization is difficult Main methods of linear optimization Line search Direction of descent Quality of approximation Center of gravity method Ellipsoid method Interior point methods Chapter 7. "Advanced Algorithms Introduction Conjugate gradient method Self-concordant barrier methods 335 Chapter 8. Economic Applications Why you should not sell your house to the highest bidder Optimal speed of ships and the cube law Optimal discounts on airline tickets with a Saturday stayover Prediction of flows of cargo Nash bargaining Arbitrage-free bounds for prices Fair price for options: formula of Black and Scholes Absence of arbitrage and existence of a martingale How to take a penalty kick, and the minimax theorem The best lunch and the second welfare theorem 386 Chapter 9. Mathematical Applications Fun and the quest for the essence 391

4 ix 9.2 Optimization approach to matrices How to prove results on linear inequalities The problem of Apollonius Minimization of a quadratic function: Sylvester's criterion and Gram's formula Polynomials of least deviation Bernstein inequality 414 Chapter 10. Mixed Smooth-Convex Problems Introduction Constraints given by inclusion in a cone Main result: necessary conditions for mixed smooth-convex problems Proof of the necessary conditions Discussion and comments 432 Chapter 11. Dynamic Programming in Discrete Time Summary Introduction Main result: Hamilton-Jacobi-Bellman equation Applications to concrete problems Exercises 471 Chapter 12. Dynamic Optimization in Continuous Time Introduction Main results: necessary conditions of Euler, Lagrange, Pontryagin, and Bellman Applications to concrete problems Discussion and comments 498 Appendix A. On Linear Algebra: Vector and Matrix Calculus 503 A.I Introduction \ 503 A.2 Zero-sweeping or Gaussian elimination, and a formula for the dimension of the solution set 503 A.3 Cramer's rule 507 A.4 Solution using the inverse matrix 508 A.5 Symmetric matrices 510 A.6 Matrices of maximal rank 512 A.7 Vector notation 512 A.8 Coordinate free approach to vectors and matrices 513 Appendix B. On Real Analysis 519 B.I Completeness of the real numbers 519 B.2 Calculus of differentiation 523 B.3 Convexity 528 B.4 Differentiation and integration 535 Appendix C. The Weierstrass Theorem on Existence of Global Solutions 537

5 x C.I On the use of the Weierstrass theorem 537 C.2 Derivation of the Weierstrass theorem 544 Appendix D. Crash Course on Problem Solving 547 D.I One variable without constraints 547 D.2 Several variables without constraints 548 D.3 Several variables under equality constraints 549 D.4 Inequality constraints and convexity 550 Appendix E. Crash Course on Optimization Theory: Geometrical Style 553 E.I The main points 553 E.2 Unconstrained problems 554 E.3 Convex problems 554 E.4 Equality constraints 555 E.5 Inequality constraints 556 E.6 Transition to infinitely many variables 557 Appendix F. Crash Course on Optimization Theory: Analytical Style 561 F.I Problem types 561 F.2 Definitions of differentiability 563 F.3 Main theorems of differential and convex calculus 565 F.4 Conditions that are necessary and/or sufficient 567 F.5 Proofs 571 Appendix G. Conditions of Extremum from Fermat to Pontryagin 583 G.I Necessary first order conditions from Fermat to Pontryagin 583 G.2 Conditions of extremum of the second order Appendix H. Solutions of Exercises of Chapters Bibliography 645 Index 651