Constrained optimization with equality constraints: a didactic approach of Lagrange's Theorem

Transcription

1 Constrained optimization with equality constraints: a didactic approach of Lagrange's Theorem Sebastian Xhonneux Introduction Because of its many uses, the constrained optimization problem is presented in most calculus courses for mathematicians but also for economists. In fact, constrained optimization can be seen as one of the fundamental technique that economists use to solve economic problems. Since the Theorem of Lagrange and the consequential method of Lagrange multipliers (named after Joseph- Louis Lagrange) provides an appealing strategy for finding the maxima and minima of a function subject to equality constraints, we are interested in studying the teaching of this theorem in both branches of study, mathematics and economics. After having presented the aforesaid theorem in its mathematical environement in the first paragraph, we give, in the second paragraph, some information about the teaching of this notion and more precisely about its proving. Then, paragraph 3 presents the theoretical framework of our didactic research. In paragraph 4, we use this framework to analyse one of the proofs of Lagrange's Theorem and explain some consequences on students' learning stemming from the particular presentation. Finally, in the last paragraph, we present the outlook of our work to which this study can lead to allow a better teaching of the mathematical object. 1 Lagrange's Theorem We have some (differentiable) function that we want to minimize (or maximize). We also have some restrictions on which points in we are interested in. The points which satisfy the constraints are refered to as the feasible region. Lagrange's Theorem deals with problems where the feasible region can be described only in terms of equality constraints of the form. Our problem is mathematically defined as follows where is a function on a certain open set and is a function verifying that is a function. Then, the feasible set is given by The Theorem of Lagrange is stated below. Theorem 1.1 (Lagrange's Theorem) Suppose is a local maximum or minimum of optimization problem. Suppose also that 1 Then, there exists a vector such that 1 We use the following notations: for the rank of a matrix and for the Jacobian matrix of function. 1

2 The scalars are called Lagrange multipliers. Note that Lagrange's Theorem only gives a necessary optimality condition for a local optima, and, at these points, only for those local optima which also meet the rank condition. Therefore these conditions can't claim that if we have a point satisfying the equation for some then the point must be a local minimum or maximum, even if also meets the rank condition. Speaking of sufficient optimality conditions, it is possible to extend the classical second derivative test to constrained optimization problems. 2 The teaching of Lagrange's Theorem The analysis of programs and textbooks shows that the first order necessary conditions for a local minimum or maximum are studied with recourse to different approaches. The teaching's contents are not always the same between universities and even from a professor to another one. Thus, for the constrained optimisation with equality contraints professors mainly privilege three approaches that we classify as follows: Intuitive and graphic approaches: Intuitions and central ideas of Lagrange's Theorem are provided by means of graphics and observations of what is actually going on when optimizing. To visualize the situation, key ideas of the proof are developed and sketched by means of geometric arguments (parallelism and perpendicularity, differentiable paths lying on a surface), algebraic arguments (linear combination of vectors, orthogonal projection on subspaces, subspaces spanned by a set of vectors, linear independence of vectors) or arguments from differential calculus (first order necessary optimality conditions for unconstrained optimization problems, Taylor Approximation, Implicit Function Theorem). This approach often chooses the case of and to start with before generalizing to the case of more than one constraint. The Implicit Function Theorem is frequently involved without giving further details. The latter is the reason why authors only give a sketch of the proof rather than a formal and rigorous proof stating that it is no laughing matter or beyond our reach 2. Intuitive approaches can be found in [6] or [12]. Calculus approach: Using a rigorous analytical style, the two most well-known proofs of Lagrange's Theorem are the elimination approach (derivation from the Implicit Function Theorem) and the penalty proof (disregard of the constraints while adding a high penalty to the objective function for violating them). Calculus approaches can be found in [3], [10] and [1]. Geometric approach: Many authors prefer presenting the proof of Lagrange's Theorem in a geometric language. In fact, the proof can give useful geometric insight into the interpretation of the first order necessary condition given by Lagrange's Theorem. Using, for example, the tangent cone permits describing the feasible set in the neighborhood of independently of its functional description. Arguments in geometric approaches are principally coming up from algebraic and differential geometry. Note that, once more, the use of the Implicit Function Theorem can be avoided if wished. We find geometric approaches in [5], [7] and [9]. For the constrained optimization course, we can notice that proofs using the Implicit Function Theorem remains widely dominant in the textbooks and notes due to its attractiveness in 2 Expressions used by J. van den Heuvel in [12]. 2

3 multivariable calculus. Nevertheless, few professors have recourse to proofs kept off this theorem considered as elaborate and cryptical 3. Like our other studies affirm, intuitive, graphic and calculus approaches often go with studies in economics while the calculus and geometric approaches are privileged for studies in mathematics. There also is an interconnection between the approach chosen and the course's content and objectives. Finally, we can point out that economists seem to attach more importance to the interpretation of Lagrange multipliers, one of the appealing focuses of the method of Lagrange. 3 Didactic Framework The purpose of the next section is to present one of the manifold proofs of Lagrange's Theorem and to analyze it from a didactic point of view. Here, we explicate the didactic framework of our work. The teaching-learning process regarding the proof of Lagrange's Theorem is a challenge to be faced when coming from unconstrained to constrained optimisation problems. As we sustain the assumption that change of settings (as intended by Régine Douady) and change of registers of semiotic representation (as intended by Raymond Duval) are indispensable to the mathematical activity, we take the liberty of regarding each step of the proof presented in appendix A with regard to the coordination of settings and registers. First, let us present the definition of a setting. According to Douady, the term setting has to be taken in its common sense. Hence, we talk about the algebraic setting, the analytic setting or the geometrical setting if we consider mathematical definitions, theorems, expressions or developments arising from algebra, from calculus or from geometry respectively. On the other hand, the graphic setting is mentioned if graphics try to illustrate a particular situation. Definition 3.1 (Setting (Douady [2], 1986)) A setting is defined as a set of the objects of a branch of mathematics, relations between these objects, and mental images associated with these objects and relations. The author underscores that the change of settings is a means to obtain different formulations of a problem which allows both a new access to the encountered difficulties and the implementation of tools and techniques which were not tangibles in the first formulation (Douady [2]). Secondly, we introduce the concept of register of semiotic representation which has direct connections with the concept of setting. We are in front of the following paradox when speaking about the learning process. (...) on the one hand, the learning of mathematical objects cannot be other than a conceptual learning and, on the other hand, it is only by means of semiotic representations that an activity on mathematical objects becomes possible (Duval [4]). In fact, research on visualisation of mathematics and the role of mental images have shown the importance of mathematics representations on the learning of mathematics. In mathematics we do not work directly with objects (i.e. with concepts), but with their semiotic representations. So, semiotics, both in mathematics and in mathematics education, is fundamental. According to Duval, 3 Expressions used by students within our practical experimentations at Belgian universities. 3

4 a semiotic system must allow to accomplish three cognitive activities. He provides the following definition. Definition 3.2 (Register of semiotic representation (Duval [4], 1993)) A semiotic system is a \emph{register of semiotic representation} if it permits three cognitive activities related to the semiosis 4 : 1. identification of the representation in the system, (e.g.: in the algebraic register, is conform but not ) 2. treatment of the representation that is the transformation of the representation in the same register where it was formed, (e.g.: in the algebraic register, the rule can be used to write ) 3. and conversion of the representation that is the transformation of the representation in another register where it conserves the totality or part of the meaning of the initial representation. (e.g.: in the algebraic register, we can transform into «the difference of two squared numbers and» in the natural language) The natural language, the symbolic languages, graphs, geometric figures, etc. are examples of registers of semiotic representation and the mathematical activity implies the coordination of these different registers of representation. Nevertheless, one keep in mind the professor who is unaware of semiotics may cherish the illusion that, if the student manipulates a concept's representation, then he is manipulating the concept itself and thus the cognitive construction has taken place. Or maybe, the student has only learnt to manipulate the semiotic representations but has not at all constructed the concept. There is a need of awareness on the teacher's side. 4 Proof of Lagrange's Theorem We are now able to present our analysis of one of the most common proof of Lagrange's Theorem depending basically on the Implicit Function Theorem. The complete proof is left as an appendix to the reader and can be found in Appendix A. Although the proof is not too long, it appears to many undergraduate students sophisticated, and we may give some hints why it is like that. Furthermore, we would like to advert the interested reader that due to shortness of the paper only one of the multiple proofs of Lagrange's Theorem is presented whereas the doctoral thesis will include the others. 4.1 Attendant settings and registers According to our classification, the presented proof arises from a calculus approach using the Implicit Function Theorem. It principally uses concepts and notations practiced in multivariable calculus courses like the gradient or the Jacobian matrix, the Implicit Function Theorem and first order necessary optimality conditions for unconstrained problems. Therefore, this approach is more suited for a presentation of Lagrange's method as a consequence of the Implicit Function Theorem than a thoughtful geometric presentation of the fact that the constrained optimization problem is a natural extension of the unconstrained case. Let us be more precise. In the theorem's specifications we find four elements: 4 Semiosis is any form of activity, conduct, or process that involves signs, including the production of meaning. Semiotics is the study of sign processes (semiosis). 4

5 1. definitions of objective function and feasible set (i.e. constraints) through the mention of optimization problem in the analytic setting, 2. supposition of the point to be a minimum or maximum of optimization problem in the natural language and in the analytic setting (hiding the mathematical definition of an optimum in the analytic setting), 3. supposition of in the algebraic setting (hiding the full rank supposition in the natural language or the linear independence supposition of gradient vectors in the algebraic setting), 4. consequence of the implication,, in the algebraic setting (hiding the fact that the vector and the set of vectors are linearly dependent in the algebraic setting). The latter three points are expressed by means of the symbolic language. Consequently, we await that a proof of Lagrange's Theorem is written in the mentioned settings manipulating elements and propositions from linear algebra, from vector and matrix calculus, from differential calculus, etc. The first paragraph of the proof stays in the algebraic setting because precisions about vectors and matrices used in the proof are given. The second paragraph, in return, comes back to results from calculus: the Implicit Function Theorem and the chain rule for differentiating the two members of the identity are emphasized. It becomes clear how simply mathematicians switch from one setting to another because they have already conceptualized the mathematical object. In fact, we pass from the algebraic setting (manipulating equations) to the analytic setting (the equation implicitely defines a function). Furthermore, the equation written down in symbolic language is the consequence of applying the chain rule in the natural language. Change of register happens. Next, we find is invertible by assumption which is a change of register from the symbolic language to the natural one too. Students should be able to understand the equivalence between a full rank square matrix and an invertible matrix. The inverse of the aforementioned matrix then becomes the key element of the 's definition. Having defined, the author shows that meets the required conditions using algebraic operations and stays therefore in the setting of linear algebra. First, the condition is proved for the variables only, and secondly for the variables only. The first part directly follows from the definition of while the second part needs a further definition of a new function,, and hence change of setting. We use, again, the chain rule for differentiate a multivariable function before applying the first order necessary optimality condition for unconstrained optimization problems in the analytic setting. Last, we go back to the algebraic setting to get the expected conclusion. 4.2 Didactic analysis The present listing of settings and registers used in the proof of Lagrange's shows that changes of settings and registers ineluctably happens. However, we make the following hypothesis: 5

6 Teachers often overestimate the student's ability to jump from one setting to another without problems believing that the students follow. Or, the teacher probably does so because he has already conceptualized while the student tries to appropriate the knowledge. In fact, two different aspects of the same mathematical object may have been presented in two different settings exclusive of the theorem, the property, etc. which coherently links these aspects. The first understanding of a proof therefore can be facilitated when presented in a number of settings as few as possible. Nevertheless, we also should keep clearly in mind that deeper insight of a mathematical concept only comes from one's capacity to create links between different settings. Teachers often underestimate the advantage of showing different semiotic representations. As the student has to conceptualize by basing himself on the semiotic representatives, he follows at the level of these representations, but not necessarily of meanings. Teachers therefore should be aware that the student's learning is aimed at the conceptualizing of mathematical objects by the use of different semiotic representations. 5 Upshot To get deeper insight in the functioning of Lagrange's Theorem and its manifold proofs, we are going to analyze the other approaches didactically. In spite of the analytic and algebraic character of the calculus approach presented in this article, it already involves several changes of settings and registers which have to be handled for a profound understanding of the proof and which we expect encountering in the other approaches. As Duval and Douady point out, changes of settings and registers help considerably to the teaching and the learning of an object. We therefore argue that the restriction of the teaching of Lagrange's Theorem only as consequence of the Implicit Function Theorem and without complementary (geometric) approaches leads to some misconceptions and difficulties of the constrained optimization problem. In particular, depending on the restriction to the algebraic setting (and analytic setting), the proof can't benefit from the enriching meaning of the geometric or the graphic approach. 6

7 References [1] D.P. BERTSEKAS, Nonlinear Programming : Second Edition, Athena Scientific, Belmont, USA, [2] R. DOUADY, Jeux de cadre et dialectique outil-objet, Recherches en didactique des mathématiques, n 7.2, 1986, pp [3] J. DOUCHET AND B. ZWAHLEN, Calcul différentiel et intégral 2 - Fonctions réelles de plusieurs variables réelles, Presses polytechniques romandes, [4] R. DUVAL, Registres de représentation sémiotique et fonctionnement cognitif de la pensée, Annales de Didactique et de Sciences Cognitives de l'irem de Strasbourg, n 5, 1993, pp [5] J.CH. GILBERT, Optimisation Différentiable - Théorie et Algorithmes, syllabus de cours à l'ensta, [6] D. KLEIN, Lagrange Multipliers without Permanent Scarring, Electrical Engineering and Computer Sciences, Berkeley, [7] A.P. KNOERR, A Dynamical Proof of the Method of Lagrange, Society for Industrial and Applied Mathematics, Volume 40, Issue 4, , [8] J.-L. LAGRANGE, Théorie des fonctions analytiques contenant les principes du calcul différentiel, Troisième édition revue et suivie d'une note, Imprimerie de Mallet- Bachelier, Paris, [9] D. LUENBERGER, Linear and Nonlinear Programming, Second edition, Kluwer Academic Publishers, Boston/Dordrecht/London, [10] E.J. MCSHANE, The Lagrange Multiplier Rule, The American Mathematical Monthly, Vol. 80, No. 8, 1973, pp [11] R.K. SUNDARAM, A First Course in Optimization Theory, Cambridge University Press, [12] J. VAN DEN HEUVEL, Notes 6 - Constrained Optimisation with Equality Constraints - Lagrange's Theorem and Method, Course notes of MA208 Optimisation Theory, The London School of Economics and Political Science, A Proof of Lagrange's Theorem We will carefully prove the Theorem of Lagrange by using the Implicit Function Theorem. The structure of the proof is based on the book ([11], 1996) A First Course in Optimization Theory de R.K. Sundaram. Notations have been adapted for the good of coherence with our work. Proof : Without loss of generality, we assume that the submatrix of that has full rank is the matrix consisting of the first rows and columns. We will denote the first coordinates of a vector by and the last coordinates by, i.e. we write. 7

8 In particular, we shall write to denote the local optimum. Furthermore, we shall treat the vector, whose existence we are to show, as a matrix. Thus, for instance, we will write to represent. As a first step, note that since, the Implicit Function Theorem applied to shows that there exists an open set in containing, and a function such that and for all. Differentiating the identity with respect to by using the chain rule, we obtain where (respectively ) denotes the derivative of at with respect to the variables (respectively variables) alone. At, we have. Since is invertible by assumption, this implies Now define by We will show that so defined meets the required conditions. Indeed, it follows from the definition of that when both sides are postmultiplied by, we obtain which is the same as Thus, it remains only to be shown that To this end, define the function by. Since has a local maximum (respectively minimum) at, it is immediate that has a local maximum (respectively minimum) at. Since is open, is an unconstrained local optimum of, and the first order conditions for an unconstrained optimization problem imply that, or by using the chain rule Substituting for from equation, we obtain which, from the definition of, is the same as Thus and Lagrange's Theorem is proved. 8