21-268: Multi-dimensional Calculus Spring 2012

Transcription

1 -68: Multi-dimensional Calculus Spring Department of Mathematical-Sciences, Carnegie Mellon University. Copyright 5, Department of Mathematical Sciences, Carnegie Mellon University. This work is licensed under the Creative Commons Attribution-Non Commercial-Share Alike 4. International License. This means you may adapt and or redistribute this document for non commercial purposes, provided you give appropriate credit and re-distribute your work under the same licence. To view the full terms of this license, visit creativecommons.org/licenses/by-nc-sa/4./ or send a letter to Creative Commons, PO Box 866, Mountain View, CA 944, USA.

2 Preface ii Preface These notes were typed in when -68: Multi-dimensional Calculus was taught by Russell Schwab in the Department of Mathematical-Sciences at Carnegie- Mellon university. The notes were typed by Christopher Almost. These notes are being distributed by the Department of Mathematical-Sciences as a service to the community, under the Creative Commons Attribution-Non Commercial- Share Alike 4. International License. This means you may adapt and or redistribute this document for non commercial purposes, provided you give appropriate credit and re-distribute your work under the same licence. This means:. You may use these as lecture notes for a similar course you are teaching, provided you follow the licence terms.. You may give these notes to your students as a references, provided you follow the licence terms. 3. You may edit these notes, and redistribute your modifications, provided you do so under the same licence. 4. If however, you would like to use material in these notes for a commercial book, you must obtain written prior approval. To view the full terms of this license, visit licenses/by-nc-sa/4./ or send a letter to Creative Commons, PO Box 866, Mountain View, CA 944, USA. The L A TEX source for these notes is currently publicly hosted at GitLab: https: //gitlab.com/gi4/cmu-math-68. If you are interested in modifying these notes, please contact the current maintainer to discuss the best method. These notes are provided as is, without any warranty and Carnegie Mellon University, the Department of Mathematical-Sciences, nor any of the authors are liable for any errors.

3 Contents Preface Contents ii iii Linear algebra review Functions of several real variables. Examples of multi-dimensional functions Notation and basic definitions Limits and continuity Differentiation of multi-dimensional functions 5. Best affine approximation and partial derivatives Linearization and the Jacobian matrix Rules of Differentiation Geometric applications 3. Curves and paths Paths on surfaces Higher derivatives and Taylor s theorem 3 4. Higher derivatives Extrema and the second derivative test Constrained extrema and Lagrange multipliers Multi-dimensional integrals 3 5. Double integrals Triple integrals Riemann sums Changes of coordinates Polar coordinates Spherical Coordinates Integrals over paths and surfaces 3 6. Line and path integrals Parameterized surfaces Surface integral Vector Calculus Divergence and curl Orientation of boundaries Flux and Gauss theorem Stoke s theorem Proof of Stoke s theorem for a graph iii

4 Contents iv 7.6 Gauss theorem Index 48

5 Linear algebra review Linear algebra review. (a) Recall that λ and v are respectively an eigenvalue and associated eigenvector of the linear map T if... If v and T v λv. (b) Implicitly, assumptions were made in (a) about the domain and range of T. What were they? They must be the same vector space.. Find the eigenvalues of A 5 4. They are and 4 because A is upper triangular and those are the diagonal elements. 3. Let p n be a fixed, non-zero, vector. (a) What space should v be in for p v to make sense? The dual space of n, which is canonically isomorphic to n. (b) What is the solution space of all v with p v? Is it a vector space? Why? What is the dimension, and why? It is the hyperplane of all vectors perpendicular to p, a vector space of dimension n. 4. (a) To which diagonal matrix, say B, is A in question similar (conjugate)? It is similar to B 4 because its eigenvalues are distinct and hence A is diagonalizable. (b) What does it mean for two matrices to be similar (conjugate)? A is similar to B if there is an invertible matrix M (a change of basis) such that A M BM. 5. Suppose D is similar (conjugate) to C. Evaluate det(d) and tr(d). They are and, respectively, because those are the answers for C, and both det and tr are invariant for conjugation. 6. Does it make sense to talk about eigenvectors of E. Why or why not? No, because the domain of E is 4 but its range is 3, which are not the same space. Functions of several real variables. Examples of multi-dimensional functions We know how to do calculus for functions f : or f : [a, b] of a single real variable. Here are some examples of functions with more inputs and outputs than just one.

6 .. Notation and basic definitions.. Examples.. Temperature in a physical space (e.g. this room). The input is the Cartesian coordinate of the point in the room, x (x, x, x 3 ), and the output is T(x), the temperature. Here T : 3 or T : 3.. Topographical height on a map or on the Earth s surface. The input is the coordinates x (x, x ), or longitude and latitude, and the output is the height, h(x). Here h : or h : S, where S is the -dimensional sphere. 3. Electrostatic potential Φ and electric field E. The input is the coordinates, x, of a point in a physical space and the outputs are Φ : 3, the scalar electric potential (voltage) at x, and E : 3 3, the vector indicating force on a particle due to electric field. We will see that, typically, E DΦ, the gradient of Φ, a topic of this course. 4. From economics, the utility function U. The inputs are the amounts of each good you could consume (e.g. hours at a pinball machine, number of apples, number of oranges, amount of fizzy beverage, etc.) The ouput is the scalar quantity that is the amount of satisfaction derived from that particular choice of consumption bundle. Here U : n +, where n is the number of goods under consideration. + [, ) is the collection of non-negative real numbers.. Notation and basic definitions We need to define some notation that makes analogy with the idea of a function of a single variable f : [a, b] in -dimensional calculus. The underlying space for this course is n, n-tuples of real numbers, with canonical basis {e,..., e n }, where e i is the n-vector with a in the i th position and zeros everywhere else. The inner product on n is defined by x, y : x y x y + + x n y n. This defines a distance on n defined by d(x, y) : x y, x y, the squared length of x y. We will also write x y : d(x, y) (absolute value bars). So the norm on n is defined by x x, x. Yet otherwise said, n x y d(x, y) (x i y i ). An ɛ-neighbourhood of x n is B ɛ (x) : {y n : d(x, y) < ɛ}, the ball centred at x with radius ɛ, not including the boundary (i.e. the open ball). A subset n is an open set if for all x, there is some neighbourhood of x also contained in (i.e. there is ɛ > such that B ɛ (x) ). This generalizes the idea that if x (a, b) then there is always room between a and x and between x and b. A subset n is closed set if its complement c : n \ is an open set. In symbols, for all y / there is some ɛ > such that B ɛ (y). i

7 .3. Limits and continuity 3.. Example. : {x : x and x } is closed. It is the closed box/square [, ] [, ]. For any y outside of (so y > or y > ) there is ɛ > such that B ɛ (y) does not intersect the square. A subset n is bounded if there is some R > such that B R (). (Note that you should be able to tell that R is a scalar from the context: the only collection in this course for which we will define a total ordering < is.).. Examples.. : {x n : x i for all i,..., n} is bounded. Taking R : 8 shows this.. : {x : x x < } is the region between two hyperbolas in. is not bounded. Indeed, it contains the line x x. 3. Vector subspaces of n are not bounded sets. In general they are not open sets either. A subset n is connected (actually path-connected) if, for all x, y, there is a path γ from x to y. (A path γ is a continuous function γ : [a, b] n, and γ is said to go from x to y if γ(a) x and γ(b) y.) For open sets, it suffices to replace path γ in the definition of connected with an ordered collection of unbroken straight line segments starting at x and ending with y. We will say that is a domain if it is open and (path-)connected. Domains are the sets over which we will do multi-variable calculus...3 Example. Consider the following function f : \ {} defined by f (x, x ) : x x x + x. What does the graph of f look like? Is f well-behaved near x? We have several tools at our disposal. The level curves of f are the sets {x \ {} : f (x) c} for various choices of c. The level set for c is the pair of lines x x and x x (not including zero). The level sets for c and are the lines x and x, respectively. In polar coordinates, f (r, θ) (r cos θ) (r sin θ) r cos(θ). Interestingly, this formula does not depend on the radius..3 Limits and continuity This section follows K.4. We saw that the function from Example..3 was not so well-behaved for its argument near zero, in the sense that the values the function takes on a circle centred at the origin are cos(θ), irrespective of the radius. Continuity is the property of a function, say f, that the values f (x) can be made as close to f (x ) as we like by taking x close to x. The precise definition is as follows. Let f : n m be a function.

8 .3. Limits and continuity 4.3. Definition. We say that lim x x f (x) c, that f has a limit of c at x or x on the boundary of, if for all ɛ > there is δ > such that f (x) c < ɛ for every x such that < x x < δ. Note that we do not require that this hold at x x. This is the δ-ɛ definition of limit. We say that f is continuous at x if lim x x f (x) f (x ). If the domain of f is then there are only two directions from which to approach a point x from the left or from the right. In n there are many ways to approach a point, and more than just the n ways along the coordinate directions..3. Example. Let f be as in Example..3 and define f (x), x g(x), x. Let s try x x for a few different -dimensional approaches. Let for the following choices of γ.. If γ(s) (s, ) then. If γ(s) (, s) then 3. If γ(s) (s, s) then γ(s) (x (s), x (s)) s lim g(γ(s)) lim s s s. s lim g(γ(s)) lim s s s. lim g(γ(s)) lim s s s. 4. If γ(s) (s cos s, s sin s) then lim s g(γ(s)) lim s cos(s). 5. Think of a function γ for which lim s g(γ(s)) does not exist..3.3 Exercises.. Let γ(s) (s cos( s ), s sin( s )) and s(). Describe the path γ. Is is continuous at s? Yes, it is continuous at s because lim γ(s) lim s cos ( s s s ), s sin ( s ) lim s γ(). s The path γ spirals around (, ), making infinitely many complete turns around it before reaching zero.. What happens with f (x) (x x )/(x + x ) under s f (γ(s))? Does lim s f (γ(s)) exist? The limit does not exist because f (γ(s)) cos( s ), which is very badly behaved near s.

9 Differentiation of multi-dimensional functions 5 The key point is that -dimensional limits existing for various ways of taking x to x (e.g. γ(s) x as s ) do not imply that the limit exists according to the n definition. Recall that if f : and g : m then g f : m : x g(f (x)) is the composition..3.4 Theorem. If f is continuous at x and g is continuous at y f (x ) then g f is continuous at x. A similar statement, appropriately modified, holds for limits. If lim x x f (x) c and lim y c g(y) l then lim x x g f (x) l..3.5 Corollary. If lim x x f (x) c and γ : [, ɛ] is any path with values in such that lim s γ(s) x then lim s f (γ(s)) exists and is equal to c. Going back to Example.3., g is not continuous at x because the limit lim x g(x) does not even exist by the corollary (because there are multiple different limits of g(γ(s)) for different choices of γ) and therefore there is no hope of lim x g(x) g(). Differentiation of multi-dimensional functions. Best affine approximation and partial derivatives Recall that, when f : is differentiable, f (x ) lim x x f (x) f (x ) x x. This implies that c f (x ) is the unique number such that lim f (x) f (x ) c(x x ) x x x x That is, the best affine approximation of f at x is l x (x) : f (x ) + c(x x ). What should we do for f : n? Could we use an affine approximation to f? This is related to the material in K.6 and K.7. Recall that a linear map n is characterized by p n, namely x p x is linear, and this gives all linear maps. The best linear approximation for f at x had better at least match at x, i.e. we should take l x (x ) f (x ). It follows that l x (x) : f (x ) + p (x x ) for some p n. But which one? The best approximation will correspond to the p such that f (x) l x (x) goes to zero faster than (x x ) as x x. Our goal is hence the unique p such that f (x) (f (x ) + p (x x )) lim. () x x x x

10 .. Best affine approximation and partial derivatives 6 But how do we do this? Note that x p x is a linear map n. To know p, we need to know p i for i,..., n. But p e i p i, so we need only look in the directions of the canonical basis. Whence, for small h, l x (x + he i ) f (x ) + p (x + he i x ) f (x ) + hp i, From the definition of the derivative, with x x + he i, It follows that f (x) (f (x ) + p (x x )) lim x x x x f (x + he i ) (f (x ) + hp e i ) lim h h lim f (x + he i ) f (x ) h p i h p i lim h f (x + he i ) f (x ) h This is the -dimensional derivative of the function g(h) : f (x + he i ). Think of g as the restriction of f to the -dimensional space x + span({e i })... Theorem. If f has l x as its best linear approximation at x in the sense of () then p i lim h h (f (x + he i ) f (x ))... Definition. The partial derivative of f : n at x in the x i component is g (), where g(h) : f (x + he i ). It is denoted f f (x + he i ) f (x ) (x ) lim. x i h h How to compute the partial derivative? The function g only records changes in f the direction of e i, so from the point of view of g the x j for j i are constants. Apply the -dimensional rules assuming that x j for j i are constant. All this material matches the material in K.5, K.6, and K Example. If f (x) : x sin(x ) + x x + x then f f x x cos(x ) + x + x. x x sin(x ) + x and The best affine approximation to f : n m can be represented by an m n matrix with respect to the canonical bases as follows. Note that f (x) is a vector in m, so think of f (f,..., f m ) T, where f i :. We require l x (x) : f (x ) + A(x x ), where f (x) (f (x ) + A(x x )) lim. () x x x x

11 .. Linearization and the Jacobian matrix 7 If this holds then A is unique and A (a i j ) i,...,m,j,...,n, where a i j [Ae i ] j is the i th component of the vector Ae j m. To isolate a i j we perform the same basic procedure as in the n case, but applied to the i th entries of l x and f. [f (x + he j ) f (x ) A(x + he j x )] i [f (x + he j ) f (x ) hae j ] i f i (x + he j ) f i (x ) [hae j ] i But this gives exactly the partial derivative of f i in the x j direction, i.e. for i,..., m and j,..., n. a i j f f i i (x + he j ) f i (x ) lim x j h h. Linearization and the Jacobian matrix We say that f : n m is total differentiable or full differentiable at x if there is an m n matrix A such that () holds. We write D f (x ) A. This matrix is also known as the Jacobian of f at x... Theorem. If f is full differentiable at x then f i x j (x ) exists and is equal to [D f (x )] i j, the (i j) th entry of A. But when is A given by the partials?.. Theorem. Let f : n m, x, and ɛ > be such that B ɛ (x ). If f i x j all exist and are continuous on B ɛ (x ) then f is full differentiable at x and [D f (x )] i j f i x j (x ). Note that m controls the norms of each of the components of f m and also of x n. So () is very strong. The information about f i (x + he j ) as functions is not enough to get the full derivative. But the extra requirement that f i x j exist and are continuous on a whole neighbourhood is enough...3 Example. lim x cos( x ) does not exist. If the limit did exists, say was c, then c [, ] because that is the range of the function cosine. Suppose that c. Let ɛ : c >. Then ɛ > and we will show that there is no δ > such that cos( x ) c < ɛ for all x < δ. Let x n : πn. Then x n as n and cos( x n ) c cos(πn) c c > c ɛ. Therefore, no matter which δ > we try, there is n large enough that x n < δ, and cos( x n ) c > ɛ. Therefore c is not the limit, so either is the limit or the limit does not exist. But repeating the same argument with the assumption that c and x n : exist. π(n+) shows that the limit cannot be, so the limit does not

12 .. Linearization and the Jacobian matrix 8 We have seen that the existence of a best affine approximation implies that the partial derivatives exist. That is to say, given f : n and x, if there is a unique p n such that f (x) f (x ) p (x x ) lim. x x x x (this is () again), then f has partial derivatives in all directions at x. We have seen that l x (x) : f (x ) + p (x x ) is the best affine approximation to f at x. Functions f for which a best affine approximation exist are said to be differentiable. Not all functions are differentiable. Taking x x + he j and plugging it into (), lim x x f (x) l x (x) x x lim h f (x + he j ) f (x ) p (he j ) he j f (x + he j ) f (x ) hp e j lim h h f (x lim + he j ) f (x ) hp e j h h f (x + he j ) f (x ) lim p e j h h f (x + he j ) f (x ) p e j lim h h x x he j e j ratio of real numbers This tells us both that the limit on the right exists and that it is equal to p e j p j. We write D f (x ) p and f x j (x ) p j. We call D f the gradient of f at x (but only when f maps to, in which case D f is an n-vector). The best affine approximation to f at x can hence be rewritten as l x (x) f (x ) + D f (x ) (x x ) f (x ) + f (x )(x x ) + + f (x )(x n x n ). x x n..4 Example. Is f (x) : x x + + x n differentiable at x? It is not, because if it were then f x () would exist, but f ( + he ) f () h h h ± where the ± depends on the sign of h. Therefore the limit as h does not exist, so the limit defining f x () does not exist. This implies that f is not differentiable. Similarly, g(x) x + x is not differentiable at x because f x () does not exist (though in this case f x () does exist).

13 .3. Rules of Differentiation 9 In which direction does f increase the fastest from f (x )? Hint: first think of which direction l x increases the fastest. Since f and l x are very close in a neighbourhood of x, the answers should be the same. The tangent plane to the graph of f : n at x is the graph of the best affine approximation l x. It is the unique hyperplane that touches the graph of f at x and which satisfies error(x x )/ x x as x x..3 Rules of Differentiation As with -dimensional derivatives, there are a few rules that the derivative satisfies.. If f : n m and c then. If f, g : then 3. If f, g : m then D(c f )(x ) cd f (x ). D(g f )(x ) g(x )D f (x ) + f (x )Dg(x ). D(f + g)(x ) D f (x ) + Dg(x ). 4. If g : l m and f : n then D(f g)(x ) D f (g(x ))Dg(x ). The last rule is the chain rule. It is one of the best reasons for using the best affine approximation definition of the derivative. g(x) g(x ) + Dg(x )(x x ) + err g (x x ) f (g(x)) f (g(x )) + D f (g(x ))(g(x) g(x )) + err f (g(x) g(x )) f (g(x )) + D f (g(x ))[Dg(x )(x x ) + err g (x x )] + err f (g(x) g(x )) f (g(x )) + D f (g(x ))Dg(x )(x x ) + [D f (g(x ))err g (x x ) + err f (g(x) g(x ))] Furthermore, because the errors in the best affine approximation go to zero faster than x x as x x, D f (g(x ))err g (x x ) + err f (g(x) g(x )) lim x x x x lim D f (g(x )) err g(x x ) err f (g(x) g(x )) + lim x x x x x x g(x) g(x ) g(x) g(x ) x x Make the previous computation precise as an exercise. Note that we have used the fact that g is continuous at x (to know that g(x) g(x )) and also that g is

14 Geometric applications differentiable at x (to know that the ratio g(x) g(x ) / x x is bounded). This shows that D f (g(x ))Dg(x ) is the correct matrix for the best affine approximation of f g at x. Check that the shapes are in fact correct for multiplying these matrices together..3. Exercise. Suppose f : n : x x + q x. What is D f (x ) for any x? It is always q because f is affine, so it is its own best affine approximation. Think of f (x) (c + q x ) + q (x x ). 3 Geometric applications Given a direction v (so v ), how does f change at x in the straight line through x in the direction v, i.e. in the affine space x + span({v})? This is a - dimensional problem. Let g(s) x +sv and use the chain rule on the composition f (g(s)). Since Dg() v and f (g()) f (x ), the derivative is D f (x ) v. This is the directional derivative of f at x in the direction v. 3. Curves and paths Notation. C () is the collection of functions f : m such that D f (x) exists and is continuous for all x. Typically is an open connected set, i.e. a domain. Such functions are said to be continuously differentiable. A path is a map γ : [a, b] n. The terminology of continuity and differentiability carry over from what we have been discussing for the past few weeks. We will write γ : γ : Dγ for paths, when appropriate, and we refer to this n-vector as the velocity vector. curve is the image left behind after tracing a path. More precisely, it is the image of a path, Γ γ([a, b]) {γ(t) : t [a, b]}. Note the difference between γ and Γ. We say that Γ is differentiable at x Γ if there exists a continuously differentiable path γ : [a, b] Γ and t [a, b] such that both γ(t ) x and γ (t ). The tangent line is the unique line with direction γ (t ) and which touches (i.e. is tangent to) Γ at x. 3.. Examples.. Let γ : [, 4π] : t (t cos t, t sin t). The partial derivatives are γ (t) cos t t sin t and γ (t) sin t + t cos t, which are both differentiable and never both zero. Indeed, the sum of their squares is + t. Hence the image of this path is a differentiable curve. [Insert sketch of the spiral.]. Consider now the path [, ] : t (t 4, t ). It is differentiable at all t (, ), but the image curve Γ is not differentiable at zero. It has a cusp at the origin.

15 3.. Paths on surfaces [Insert image. Note that γ γ.] To prove that the curve Γ is not differentiable at (, ), we must show that any possible candidate γ : [a, b] Γ that is continuously differentiable fails the requirement that γ (t ). By shifting, we may always assume without loss of generality that t. Observe that all points x Γ satisfy x x. Therefore γ(t) (c(t), c(t) ) for some c : [a, b]. We are requiring that γ is continuously differentiable, so there is a best affine approximation at zero, call it v γ (), such that γ(t) γ() v(t ) lim t t lim t lim t (c(t) t v ) + ( c(t) t v ) t ( c(t) t v ) t c(t) t t t v lim t lim c(t) t v t Therefore we may conclude c(t) v lim t t v lim c(t) t t lim t v t lim c(t) t t c(t) lim t t c(t) lim t t x x is continuous Since γ is continuously differentiable, c () exists, and by the limit above, it must be zero. Can it be the case that c () but ( c(t) ) ()? Again, since γ is continuously differentiable, c(t) is differentiable at t. The rest of the proof is an exercise. 3. Paths on surfaces What is a surface? Intuitively, a surface is a -dimensional object inside of a 3- dimensional space, e.g. a piece of paper in this room, the surface of a ball (a sphere), etc. More precisely, S n+ is a level surface if it can be written as

16 3.. Paths on surfaces S {x n+ : F(x) c}, where F : n+ is continuously differentiable and c is a fixed constant, and DF(x) for all x near S. In this case we say the dimension of S is n. Note that F : n+ actually describing an n-dimensional surface is special. We must be careful with F, because not every such function gives rise to an n-dimensional surface. 3.. Examples.. Justification for DF. Let F(x), x ( x ), x >. Then F is continuously differentiable. (Check this carefully for x.) [Insert diagram of the 3-d picture of F.] Note {x : F(x) } {x : x } is not -dimensional in the usual sense of the term. DF(x) for all x in this level set, which is why we exclude this case.. Now let F(x) x. Then DF(x) x/ x for all n and F is not differentiable at zero. For c >, the level surface associated with F and c is the circle of radius c when n and the sphere of radius c when n If f : then the graph of f is {x 3 : x 3 f (x, x )}. It can be realized as a -dimensional surface by taking F : 3 with F(x) f (x, x ) x 3 and c. Note that DF ( f x, f x, ) is never zero. For example, if f (x) x x then F(x) x x x Another way to find curves in other than images of paths is level surfaces of F :, i.e. a -dimensional level surface is basically a curve. For example, if F(x) x s then the level surfaces are a tool for understanding the graph of F. At c we have x x, so x ±x. For c we get x x, a hyperbola, and for c, we get x x, another hyperbola. [Insert diagram of level sets and of the 3-d picture.] 3.. Example. Let γ : [, ] 3 : t (t cos(4πt), t sin(4πt), t ), and note that γ(t) for all t. This means that γ happens to always take values on the sphere S {x 3 : x }. The path of γ starts at the north pole at t and spirals down in an anti-clockwise direction until it reaches the equator at t after making two full revolutions. In general, if γ is a path on a level surface S {x : F(x) c} then, necessarily, F(γ(t)) c for every t in the domain of γ. A special case is a path on a graph of a function f : n. Recall that the graph of f is an n-dimensional surface in n+. If γ : [a, b] n+ is on the graph of f then it must be the case that γ(t) (γ (t),..., γ n (t), f (γ(t))), i.e. the (n + ) st component of γ is related to the first n components by f. What is the analog of the best affine approximation for surfaces? If the level surface is the graph of a function f, then we have seen that the tangent plane at a

17 Higher derivatives and Taylor s theorem 3 point x is exactly the graph of the best affine approximation of f at x. Basically all directional derivatives are encoded in the slope of the tangent plane. We determined the partial derivatives by reducing to the -dimensional case. For each direction e i, we consider the function g i (h) f (x + he i ), so that the partial derivatives are given by f x i (x ) g (). Note that g i is a path on the graph of f that passes through x! Furthermore, g () D f (x ) e i. This is good material with which to formulate a definition. We say that the tangent plane to the graph of f is the unique plane which touches the graph at (x, f (x )) and contains the velocity vectors, γ i (), of each of the paths γ i (s) : (x + se i, f (x + se i )). Now let S {x n+ : F(x) c} be a level surface. We say that the tangent plane (or tangent hyperplane) to S at x S is the unique n-dimensional plane that contains x and x + γ() for every continuously differentiable path γ contained in S with γ() x. Recall that γ is a path in S if γ : [a, b] n+ and γ(t) S for all t [a, b]. Equivalently, if F(γ(t)) c for all t [a, b]. From this latter equation (and the chain rule) we will be able to calculate the equation of the tangent plane. Differentiating at t, DF(γ()) γ() DF(x ) γ() But how many choices are there for γ() as γ ranges over the collection of appropriate paths? The tangent plane should contain all the directions solving DF(x ) p, or equivalently and more precisely, the tangent plane is the affine space x + (span({df(x )})). Yet otherwise said, x is in the tangent plane to S at x S if and only if DF(x ) (x x ) Example. Let S be the unit sphere in 3, S {x 3 : x }. The tangent plane at (,, ) is computed as follows. In this case F(x) x, so (compute as an exercise) DF(x) x/ x, and since x (,, ), DF(x ) (,, ). [Insert diagram?] The tangent plane is the unique plane perpendicular to (,, ) containing (,, ). (It is a coincidence of the sphere that x DF(x ).) More explicitly, the tangent plane is {(, s, t) : s, t }. 4 Higher derivatives and Taylor s theorem 4. Higher derivatives In the -dimensional case, if f : is smooth enough then f : and the derivative operation can be repeated over and over. When f : n, we compute D f via partial derivatives, which are each -dimensional derivatives. If f is smooth enough, then f x i is differentiable, so we can take further partial derivatives. Unlike the -dimensional case, there is not a unique second derivative ; there are n combinations of second partial derivatives of f. We write f x j x i for the second partial derivative of f with respect to x i first and then with respect

18 4.. Higher derivatives 4 to x j. This notation can be extended in the obvious way to k partial derivatives with respect to a list of k indices i,..., i k (in that specific order): k f x ik x i. An obvious question is, Does the order really matter? 4.. Example (Important warning). Let x x x f (x) x x +x, x, x. Away from x, f is nice and we can figure out the derivatives using the usual rules. f etc. x We will see when we fill in this example that, at some x, Mind blown. Avia f f. x x x x 4.. Theorem. If f : n and both f x i f x i x j and f x j x i and f x j both exist and are continuous then they are equal. are differentiable and if PROOF: The case is the only important one for the main idea. Let x be fixed. We will show that f x x (x) and f x x (x) are equal to the limit of the S(y) same quantity lim y y y. Assume that y is small enough that x + y. Let S(y) : f (x + y) f (x + y, x ) f (x, x + y ) + f (x). We want to isolate two separate -dimensional function and apply the mean value theorem for derivatives. Towards this goal, let Then g(s) : f (s, x + y ) f (s, x ) and h(s) : f (x + y, s) f (x, s). g(x + y ) g(x ) S(y) h(x + y ) h(x ). The mean value theorem applies, so there is x between x and x + y such that f S(y) g(x + y ) g(x ) g ( x )y y ( x, x + y ) f ( x, x ). x x

19 4.. Higher derivatives 5 Apply it again, this time to the function s f x ( x, s), to obtain x between x and x + y such that f ( x, x + y ) f f ( x, x ) y ( x, x ). x x x x f Unravelling this, S(y) y y x x ( x, x ). Applying the same reasoning to h, it f can be shown that S(y) y y x x (ˆx, ˆx ) for some ˆx i between x i and x i + y i for i,. Now we appeal to the continuity of the second partial derivatives to conclude f S(y) (x, x ) lim f (x, x ). x x y y y x x Note of course that x i x i and ˆx i x i as y i for i,. Notation. C k () : {f : all partials of f of order up to and including k exist and are continuous}. Such functions are said to be k-times continuously differentiable. Recall from -dimensional calculus that if f is better than C then we can make better than affine approximations to f. This holds for arbitrarily high orders if f has that many derivatives. In particular, if f C k then there is a function R k (x) such that R k (x) as x x and f (x) f (x ) + f (x )(x x ) + f (x )(x x ) k! f (k) (x )(x x ) k + R k (x)(x x ) k. How can we do this for f : n? For simplicity, let s consider n. We should try to apply what we already know to the single variable functions g (s) : f (x + se ) and g (s) : f (x + se ). Naively, we take the partials of g i and add them up. Is is true that f (x) f (x ) + f (x )(x x ) + f (x x x )(x x ) + f (x )(x x ) + f (x x x )(x x )? We are missing the cross terms! Obviously terms of the form x x will feature in this approximation, in general. The proof of Theorem 4.. tells us which extra terms to include, namely f x x (x )x x. We assume that f is nice enough that the mixed partials are equal.

20 4.. Higher derivatives 6 In fact, for f C, the second order approximation is f (x) f (x ) + D f (x ) (x x ) + (x x ) T D f (x ) (x x ) f (x where D f (x ) x ) f x x (x ) f x x (x ) f (x x ) Note that D f (x ) is a symmetric matrix Theorem (Taylor s theorem for order 3). Let f C 3 () and x be given. There are functions R i,j,k :, i, j, k,..., n, such that R i,j,k (x) as x x and f (x) f (x ) + D f (x ) (x x ) + (x x ) D f (x )(x x ) n i, j,k n i, j,k 3 f x i x j x k (x )(x i x i )(x j x j )(x k x k ) R i,j,k (x)(x i x i )(x j x j )(x k x k ) Aside on tangent spaces and tangent planes: There is a key distinction between tangent spaces and tangent planes. Let S {x n+ : F(x) } be an n-dimensional surface in n+. Recall that the tangent space at x is T ker(df(x )) {p n+ : DF(x ) p }. It is a linear subspace of n+ of codimension. By definition, DF(x )/ DF(x ) is a vector of length that is perpendicular to T. Note that T contains the velocity vectors γ() for all paths γ with γ() x. But T most likely does not touch T at x! The tangent plane is the translation of the tangent space, x + T {v x + p : p T}. [Insert diagram of generic S in 3 with the various planes.] Why is DF(x ) perpendicular to the surface at x? If γ is a path on S then F(γ(s)) for all s (by definition), so D(F(γ(s))), and the chain rule implies that, at s, DF(γ()) γ(). Hence DF(x ) γ(), so every vector in the tangent plane is perpendicular to DF(x ). Consider the difference between the graph of f : n+ and the level surface associated with F. The tangent plane to the graph of f at x n is the graph of x f (x ) + D f (x ) (x x ). The function that defines a level surface that is the same as the graph of f is F(x) f (x,..., x n ) x n+. For x n, the corresponding point on S {x n+ : F(x) } is (x, f (x )) n+. Then DF(x, f (x )) (D f (x ), ), so the equation defining the tangent plane T is This gives the same tangent plane. f x (x )p + + f x n (x )p n p n+.

21 4.. Higher derivatives Examples.. Give a second order polynomial approximation to f (x) sin(x x ) at x (, π/). Note that f is nice as it is the composition of smooth functions, so we must calculate D f and D f. D f (x) (x cos(x x ), x cos(x x )) D x f (x) sin(x x ) cos(x x ) x x sin(x x ) cos(x x ) x x sin(x x ) x sin(x x ) so D f (x ) (, ) and Therefore sin(x x ) + π D /4 f (x ) π/ x T π /4 x π/ π/ π/. π/ x x π/ π 8 (x ) π (x )(x π/) (x π/) As long as we are will to admit a local error of order x x (small), f is approximately equal to this quadratic in the neighbourhood of (, π/) Example (Comments on Homework 3, Question.7). Given f (x) x Ax, find a path γ : [, ] 3 such that g f γ attains a local minimum at t /. To do this, compute the eigenvalues and eigenvectors of A. As A is symmetric, there is a basis of 3 consisting of eigenvectors, {v, v, v 3 }. Without loss of generality, assume that λ >. The easiest path will be γ(t) (t t )v. In this case, f (γ(t)) (t t )v A((t t )v ) (t t ) λ v > for all t t, and it is zero at t t Example. Use the second order Taylor expansion to approximate (3.98 ) (5.97 3).

22 4.. Higher derivatives 8 To do this, we use f (x) (x ) /(x 3), x (4, 6), and x (3.98, 5.97). For the second order expansion, we need all partials up to second order. f x (x )(x 3) f x (x ) (x 3) 3 f f 4(x )(x 3) 3 x x x x From these and the Taylor expansion, f 6(x x )(x 3) 4 f (x x 3) f (x) f (x ) + D f (x ) (x x ) + (x x ) D f (x )(x x ) + 3 (.) 3 (.3) + 9 (.) (.)(.3) (.3) The exact value is.675. Note that f (x) polynomial + R(x), by Taylor s theorem, where R(x)/ x x as x x. Therefore, for this example, we expect the difference between f (x) and the approximation to be within C Examples.. Evaluate lim x ( + x ) x / x + x. Plugging in gives the indeterminate form /, so we must be more clever. Let f (x) ( + x ) x exp(x log( + x )) and apply Taylor s theorem at x. f x x + x f (x) f x log( + x )f (x) f f f (x) + x log( + x ) f (x) x x x x + x + x f x x ( + x ) f (x) + x f (x) + x f log( + x x ) f (x)

23 4.. Extrema and the second derivative test 9 We obtain f (x) + + x x + R(x) + x x + R(x), where R(x)/ x as x. This implies that the limit is zero.. Evaluate lim x ( + x ) x ( + x x ) /(x + x ). For this example, the denominator is x, so the limit is still zero. 4. Extrema and the second derivative test We say that f : n has a local minimum at x if f (x) x for all x B ɛ (x ) for some ɛ >. If f (x) > x (strict inequality) for x x then we say f has a strict local minimum at x. We say that f has a (strict) local maximum if f has a (strict) local minimum. Let s figure out what local minima of f say about D f and D f. Note that we may always focus on minima without loss of generality because differentiation is linear. Assume first that f C () and x (so that it is an interior point). Along the directions of the canonical basis, the -dimensional functions g i (s) : f (x + se i ) all attain local minima at s. This observation allows us to prove the following theorem. 4.. Theorem. If f C () and x is a local minimum of f then D f (x ). PROOF: f x i (x ) d ds g(s) s g i () for i,..., n. If f C then the previous theorem applies, of course, and if f (x ) then Taylor s theorem tells us that f (x) f (x ) + D f (x ) + (x x ) D f (x )(x x ) + R(x) (x x ) D f (x )(x x ) + R(x) The pure quadratic (x x ) D f (x )(x x ) has a local minimum at x if and only if all of the eigenvalues of D f (x ) are all non-negative. 4.. Theorem. If f C () and x is a local minimum of f then for all w n, w D f (x )w. Yet otherwise said, D f (x ) is a non-negative definite matrix. We note that the condition, w D f (x )w for all w n, is equivalent to all of the eigenvalues of D f (x ) being non-negative. PROOF: We wish to show the inequality w D f (x )w for each w. To this end, let w n be a generic fixed element with w. Because x was assumed to be interior to, there is some ε > such that B ε (x ). Thus for t small

24 4.. Extrema and the second derivative test enough (smaller than ε/ w ), x + tw and we can use Taylor s Theorem with x x + tw f (x + tw) f (x ) + D f (x ) + (tw) D f (x )(tw) + R(x + tw), and recall the very useful fact that R(x + tw)/ tw as tw. The inequality is nearly staring us in the face once we rearrange terms and use the local minimum property. Indeed, for t small enough we have and hence (recall D f (x ) ) f (x + tw) f (x ) f (x + tw) f (x ) t w D f (x )w + R(x + tw). Again rearranging, we see that we have obtained the inequality we want but with the remainder as a error: R(x + tw)/t w D f (x )w. Taking the limit as t and using the property of the remainder function, we conclude w D f (x )w. Since w was generic we conclude the theorem. Now we ask a slightly different question. We know that a local minimum forces a sign on the matrix D f. However, can we use information about D f to determine if a given point is a local minimum? The answer is yes Theorem. If x (i.e. x is an interior point of ), D f (x ), and for all w n, w D f (x )w >, then f has a strict local minimum at x Definition. We say that f C () has a saddle point at x if D f (x ) and D f (x ) has at least one positive and one negative eigenvalue. Saddle points are named for the simplest saddle point, the one at the origin of the graph of f (x) x x, which looks like a saddle Theorem. If x (i.e. x is an interior point of ), D f (x ), and D f (x ) has at least one positive and one negative eigenvalue, then f has neither a local minimum nor a local maximum at x. PROOF (SKETCH): Let v and v + be eigenvectors associated with eigenvalues λ < and λ + >, respectively. Use Taylor s theorem to investigate f (x + t v ) and f (x + t v + ). The argument from last time gives a strict local minimum/maximum for these two functions, implying that f has neither a minimum nor a maximum at x.

25 4.3. Constrained extrema and Lagrange multipliers Remark.. If f has a local minimum at x then D f (x ) and D f (x ).. If D f (x ) and D f (x ) > then x is a strict local minimum. 3. If f has a strict local minimum at x then it is not necessarily the case that D f (x ) >, i.e. there may be w such that w D f (x )w. 4. If D f (x ) has eigenvalues of differing signs then x is neither a local minimum nor a local maximum Examples.. Let f (x) e x e x. Where can f has a local minimum? Can f ever have a strict local minimum? Does f have a global minimum? We calculate D f (x) (x, x )f (x) and + 4x D f (x) 4x x 4x x + 4x f (x). D f (x) if and only if x, so the only possible point at which there can be a minimum is x. At this point, D f (), which has eigenvalues ±. Therefore f does not have a local minimum (or maximum) at x. It is not hard to see that f does not attain a global minimum.. Let f (x) e x ( e x ). Where can f has a local minimum? Can f ever have a strict local minimum? Does f have a global minimum? Again, D f (x) (x e x ( e x ), x e x e x ) and D f (x) ( + 4x )ex ( e x ) 4x x e x e x 4x x e x e x ( 4x )ex e x. Now D f (x) implies x, but x can be anything. D f (x, ) e x for any x, so all points on the line x are local minima. None of these local minima are strict. Finally, f has a global minimum value of, which is attained at all points on the line x. 3. For f (x) x 4 + x 4, D f (), but f has a strict local minimum at x. 4.3 Constrained extrema and Lagrange multipliers Going back to -dimensional calculus, to solve the problem of finding the minimizer x [a, b] of f : [a, b], we usually go through the following process:. Find all solutions to f (x) in (a, b) (the critical points).. Check which solutions of f (x) have f (x). (This step is optional.) 3. Compare the values of f (x) for all x obtained and also x a, b. The smallest gives the minimizer.

26 4.3. Constrained extrema and Lagrange multipliers The points a and b are the boundary points of the set [a, b]. In higher dimensions the boundaries are much more interesting and delicate. For n, the boundary is : {x n : for all ɛ >, B ɛ (x) and B ɛ (x)}. In the multi-dimensional settings, given a domain and f :, we are concerned with finding the minimizer for the problem min x f (x). If is bounded then is compact, so the minimum value is achieved for some x, i.e. f (x ) min x f (x). As before, we must check critical points and boundary points. Unlike in the -dimensional setting, the boundary is not simply two points; it is a whole curve! For many (actually all) of our examples, {x n : F(x) } for some function F : n. In fact, taking F(x) to be the (signed) distance from x to the boundary will do. Such Fs are not unique. Finding the minimum over the boundary is now reduced to finding the minimum of f on the set {x n : F(x) }, which leads to constrained optimization. Suppose that the minimum value of f over occurs at x. If γ : [a, b] is a path in the boundary with γ() x, then t f (γ(t)) has a minimum value of f (x ) at t. Therefore it s derivative with respect to t at t is zero. Hence by the chain rule, D f (x ) γ(), and this works for any path. We already know that γ() is perpendicular to DF(x ) for every path in. If the tangent space has codimension one (i.e. if n has full dimension ) then it must be the case that D f (x ) and DF(x ) are parallel! Therefore there is λ such that D f (x ) λdf(x ). Rephrasing, if x is a minimizer of f on then there must exist some λ such that D f (x ) λdf(x ). After introducing the new variable λ, there are n + unknowns, and the equality of vectors is n equations. But F(x ) (the fact that x ) gives another equation. Thus there is a hope of finding solutions! This is the method of the Lagrange multiplier.. Find all solutions to D f (x ) for x (the critical points).. Check which solutions of D f (x ) have D f (x ). (This step is optional.) 3. Solve for x and λ with D f (x ) λdf(x ) and F(x ). 4. Compare the values of f (x ) for all x obtained. The smallest gives the minimizer Examples.. Let f (x) x x and B (). Then {x : x } and D f (x) (x, x ). [Insert image.] D f (x) implies that x, and this is the only critical point. The system of equations obtained by introducing the Lagrange multiplier is x λx x λx x + x If x then λ and hence x and x ±. Similarly, if x then λ and hence x and x ±. There are hence five points

27 Multi-dimensional integrals 3 to compare, (, ), (±, ), and (, ±). The minimum of occurs at both (, ±).. Let f (x) x x and B (). Then D f (x) (x, x ), so x is the only critical point in. The equations resulting from introducing the Lagrange multiplier are x λx x λx x + x. It follows that λ 4 since x and x cannot both be zero. Therefore λ ±, so x ±x, and hence (±, ± ) are the four solutions. Of the five possibilities (one critical point and four points on the boundary of ), the minimum value is, which occurs at (, ) and (, ). 3. Let Γ {(t, t + ) : t }, a line in, and let f (x) x + x. Show that f does not attain a maximum value on Γ and compute the minimum value of f on Γ. Recognize that Γ is the boundary of the region {x : x + x }. To find the max/min of f on Γ we use a Lagrange multiplier. x λ() x λ( ) x + x Therefore λ, x /, and x /. So (/, /) is the only possible critical point of f on Γ. The value of f at this point is /, and it is clear that f can take larger values than this on Γ (e.g. f (, ) > /). Therefore this point is the minimum, and there is no maximum. 5 Multi-dimensional integrals Continuing our quest to extend all of the ideas from -dimensional calculus to multi-dimensional calculus, we turn to integrals. The integral, b f (x) d x, of a a continuous function f : [a, b] can be defined to be the (signed) area between graph of f, {(x, y) : y f (x)} and the x-axis. The obvious analog for f : n is the volume of the region lying between the hyperplane n, embedded in n+ as {x n+ }, and the graph of f, {x n+ : x n+ f (x,..., x n )}. When n this can be visualized and corresponds to the usual idea of (signed) volume. For n > this generalizes volume. 5. Double integrals 5.. Example. The simplest case has [a, b] [c, d], a rectangle, and f (x) α, a constant. The region between the x -x plane and the graph of f is the rectangular prism with volume α(b a)(d c).

28 5.. Double integrals Example. Let [, ] [, ] and f (x) x + x. [Insert diagram.] Consider slices of the volume obtained by holding x constant. When x s [, ] is held fixed, the height of the curve on the corresponding slice is given by the function g(x ) s + x. The area of this slice is hence g(x ) d x (s + x ) d x s + 3. If we then add up (i.e. integrate) over all slices, we can calculate the total volume to be x + d x We can write this method succinctly as x + x d x d x 4 3, where we integrate first with respect to x and then with respect to x. Would we have obtained the same answer if we integrated with respect to x first? For a general volume in 3, Cavalieri s principle of volume says that volume is equal to the integral of cross-sectional area, i.e. vol area(s) ds, where s parameterizes the cross-sectional area. This principle works, of course, for signed area as well. When [a, b] [c, d] is a rectangle, the integral of a function f : may be computed as b d a c f (x, x ) d x d x, since, as a function of x, the cross-sectional area is given by d f (x, x ) d x. We will write f (x) da for this double integral of a continuous function f :. When written with explicit bounds, it is referred to as an iterated integral Theorem. If [a, b] [c, d] and f : is continuous then both b d f (x a c, x ) d x d x and d b f (x c a, x ) d x d x exist and they are equal to each other. In this case we say that the double integral, f (x) da, exists and is equal to the common value of the iterated integrals Example. Let [, π/] and f (x) cos(x ) sin(x ). [Insert image.] c

29 5.. Double integrals 5 Using the theorem, since f is a continuous function, f da π π π cos(x ) sin(x ) d x d x sin(x ) d x π cos(x ) d x 5..5 Example. Let be a metal plate sitting in the x-y plane as [, ] [, ]. If has a non-constant density of mass modelled by ρ(x, y) ye x y g/cm, then what is the total mass of? The usual rule of mass density area only applies when the density is constant. Note that ρ is continuous, so for very small subrectangles it is approximately constant. Mass is additive, so we may compute the Riemann sums and take the limit to obtain that the mass is the integral of density. mass ρ(x, y) d xd y e 3 ye x y d xd y (e y ) d y To compute the integral over regions that are not necessarily rectangular, we appeal to Cavalieri s principle. Suppose {(x, y) : ϕ (x) y ϕ (x)} is the region between two graphs (we say that is a region of type I). Then f (x, y) da b ϕ (x) a ϕ (x) f (x, y) d yd x. There should be nothing special about this particular orientation. If we can write {(x, y) : ψ (y) x ψ (y)} then is a region of type II and f (x, y) da b ψ (y) a ψ (y) f (x, y) d xd y. There are regions that are neither type I nor type II, and it is a bit of an art to decompose them into pieces over which the integral can be computed Examples.. Let f (x, y) x + y and {(x, y) : x /, y x }. [Insert diagram, including coloured lines in both directions.]

30 5.. Triple integrals 6 This region is of both type I (as given) and type II, which can be seen by noting {(x, y) : y /4, y x /}. Hence x x f da 3 6. Evaluate x y d yd x. x [Insert diagram 3 of.] x + y d yd x 4 y x + y d xd y x 3 + x 4 d x x x 3 x y d yd x x 5 x 7 d x 48 3 y y x y d xd y. 3. As a type II integral, the above integral may be written 5. Triple integrals Consider two point objects P i in 3 with masses m i and locations (x i, y i, z i ), for i,. Then the center of mass of the two points is the weighted average of the positions, (x, y, z) m + m (m x + m x, m y + m y, m z + m z ). With n points the formulae are analogous, e.g. x m x + + m n x n m + + m n. If you had a rectangular prism [a, b] [c, d] [e, f ] in 3 with non-constant mass density ρ : then where is its centre of mass? If ρ is nice enough (e.g. continuous) then we can decompose each interval into N small pieces, hence decomposing into N 3 small cubes, over which ρ is approximately constant. Over each small rectangle the mass is approximately the volume times the (nearly) constant density. The centre of mass of is then the weighted sum of the N 3 small rectangles. In symbols, N x x i ρ(x i, y j, z k ) x i y j z k total mass i, j,k N i,j,k x iρ(x i, y j, z k ) x i y j z k as N. N i,j,k ρ(x i, y j, z k ) x i y j z k xρ(x, y, z)dv ρdv