E 600 Chapter 3: Multivariate Calculus

Transcription

1 E 600 Chapter 3: Multivariate Calculus Simona Helmsmueller August 21, 2017

2 Goals of this lecture: Know when an inverse to a function exists, be able to graphically and analytically determine whether a function is convex, concave or quasiconcave. Have an intuition about the meaning of derivatives as measures of small change, be able to calculate the Gradient, Jacobian and Hessian of a function Know all the calculation rules for derivatives (sum, quotient, product, chain rule) Know the difference between partial and total derivatives Be able to write down the Taylor approximation of a function Know what a function s differentials tell us about its continuity and convexity Know and be able to apply the calculation rules of integral theory; know and be able to apply Fubini s theorem

3 Contents Multivariate functions Invertibility Convexity, Concavity, and Multivariate Real-Valued Functions Multivariate differential calculus Introduction Revision: single-variable differential calculus Partial Derivatives and the Gradient Differentiability of real-valued functions Differentiability of vector-valued functions Higher Order Partial Derivatives and the Taylor Approximation Theorems Characterization of convex functions Integral theory Univariate integral calculus The definite integral and the fundamental theorem of calculus Multivariate extension

4 Definition (Surjective functions) A function f : X Y is said to be surjective or onto, if for every y Y there exists at least one x X with f (x) = y.

5 Definition (Injective functions) A function f : X Y is said to be injective or one-to-one, if the following property holds for all x 1, x 2 X : f (x 1 ) = f (x 2 ) x 1 = x 2.

6 Theorem ( Existence and definition of the inverse function) Let f : X Y be onto and one-to-one (i.e. bijective). Then there exists a unique bijective function f 1 : Y X with f 1 (f (x)) = x. Conversely, if such a function exists, then f is bijective.

7 Definition (Convex Real Valued Function) Let X R n. A function f : X R is convex if and only if X is a convex set and for any two x, y X and λ [0, 1] we have f (λx + (1 λ)y) λf (x) + (1 λ)f (y) Moreover, if this statement holds strictly whenever y x and λ (0, 1), we say that f is strictly convex.

8 Definition (Concave Real-Valued Function) Let X R n. A function f : X R is concave if and only if f is convex. Similarly f is strictly concave if and only if f is strictly convex.

9

10

11 Definition (Multivariate Convexity) Let X be a convex subset of R n. A real-valued function f : X R n is (strictly) convex if and only if, for every x X and every z {z R n {0} z = 1}, the function g(t) = f (x + tz) is (strictly) convex on {t R x + tz X }.

12 Definition (Quasiconvexity, Quasiconcavity) Let X be a convex subset of R n. A real-valued function f : X R is quasiconvex if and only if for any c R the set L c := {x x X, f (x) c}, also referred to as f lower-level set, is convex. Iff f is such that L + c := {x x X, f (x) c}, also referred to as f upper-level set, is convex for any c R, then f is said to be quasiconcave.

13 Theorem ( Quasiconvexity, Quasiconcavity) Let X be a convex subset of R n. A real-valued function f : X R is quasiconvex if and only if x, y X λ [0, 1] f (λx + (1 λ)y) max{f (x), f (y)} Iff f is such that x, y X λ [0, 1] f (λx + (1 λ)y) min{f (x), f (y)} then f is said to be quasiconcave.

14 Contents Multivariate functions Invertibility Convexity, Concavity, and Multivariate Real-Valued Functions Multivariate differential calculus Introduction Revision: single-variable differential calculus Partial Derivatives and the Gradient Differentiability of real-valued functions Differentiability of vector-valued functions Higher Order Partial Derivatives and the Taylor Approximation Theorems Characterization of convex functions Integral theory Univariate integral calculus The definite integral and the fundamental theorem of calculus Multivariate extension

15 Calculus is the mathematical study of change, in the same way that geometry is the study of shape and algebra is the study of operations and their application to solving equations. It has two major branches, differential calculus (concerning rates of change and slopes of curves), and integral calculus (concerning accumulation of quantities and the areas under and between curves); these two branches are related to each other by the fundamental theorem of calculus, [...] [which] states that differentiation and integration are inverse operations.

16 Theorem Let f, g : R R differentiable at x 0 with derivatives f (x 0 ), g (x 0 ), and λ, µ R. Then 1. λf + µg is differentiable in x 0 with (λf + µg) (x 0 ) = λf (x 0 ) + µg (x 0 ), 2. fg is differentiable in x 0 with (fg) (x 0 ) = (f g + fg )(x 0 ) 3. if g(x) 0 in a neighborhood of x 0, then f /g is differentiable with ( ) f (x 0 ) = f g fg g g 2 (x 0 ), 4. if all the following expressions are well-defined, then g f is differentiable in x 0 with (g f ) (x 0 ) = g (f (x 0 ))f (x 0 ).

17 Let f be differentiable at x, then f is continuous at x. Let f be differentiable at x, then there exists a good linear approximation of f in a neighborhood Let a and b be real numbers such that a < b. If f is continuous and differentiable on (a, b), then: (i) f (x) = 0 for all x (a, b) iff f is constant on (a, b). (ii) f (x) < 0 for all x (a, b) iff f is decreasing on (a, b). (iii) f (x) > 0 for all x (a, b) iff f is increasing on (a, b).

18 Convince yourself of these geometric properties by looking at the following four functions: 1. f 1 (x) = x 2 2. f 2 (x) = x 3. f 3 (x) = max{k Z : k x} 4. f 4 (x) = 1 if x Q and f 4 (x) = 0 if x R Q

19 Definition (Partial Derivative) Let X R n and suppose f : X R. If x is an interior point of X, then, the partial derivative of f with respect to x i at x is defined as f ( x) x i := lim h 0 f ( x 1,..., x i 1, x i + h, x i+1,..., x n ) f ( x 1,..., x i,..., x n ) h with h R, whenever the limit exists. Another common notation for the partial derivative of f with respect to x i at x is f i ( x).

20

21 Definition (Gradient) Let f : R n R a function which is partially differentiable with respect ot all x i, i = 1,..., n. Then the row vector ( ) f ( x) := f 1 ( x) f 2 ( x) f n ( x) is called the gradient of f at x.

22

23 Definition (Jacobian matrix) Let f : R n R m with partially differentiable component functions f 1,..., f m : R n R, and let x R n. Then the Jacobian matrix of f at x is defined as f 1 f 1 f 1 J f ( x) = ( x) x 1 f 2 ( x) x 1. f m ( x) x 1 ( x) ( x) x 2 x n f 2 f 2 ( x) ( x) x 2 x n..... f m f m ( x) ( x) x 2 x n.

24

25 Definition (Multivariate Derivative) Let X R n and suppose f : X R. If x is an interior point of X, then f is differentiable at x if and only if there exists a row vector Df ( x) such that f ( x + h) f ( x) Df ( x) h lim = 0 h 0 h where h is a vector in R n. If such a vector Df ( x) exists, we interpret it as the derivative of f at x.

26 Theorem ( Total Differential, Partial Derivative, and Gradient) Let X R n and suppose f : X R. If x is an interior point of X, and if f is differentiable at x, then: (i) all partial derivatives 1 of f exist at x and, (ii) z in X, z = 1: df ( x, z) := Df ( x) z = f ( x) z

27 Theorem ( Partial Differentiablility and Differentiability) Let X R n, suppose f : X R, and let x be an interior point of X. If all the partial derivatives of f at x exist and are continuous, then f is differentiable.

28 Definition (Multivariate Derivative of Vector-Valued Functions) Let X R n and suppose f : X R m. If x is an interior point of X, then f is differentiable at x if and only if there exists a matrix Df ( x) such that f ( x + h) f ( x) Df ( x) h lim = 0 h 0 h where h is a vector in R n. If such a matrix Df ( x) exists, we interpret it as the derivative of f at x.

29 Theorem ( Multivariate Derivative and Jacobian Matrix) Let X R n, suppose f : X R m, and let x be an interior point of X. Then, f is differentiable at x if and only if each of its component functions is differentiable at x. Moreover, if f is differentiable at x, then: (i) all partial derivatives of the component functions exist at x, and (ii) the derivative of f at x is the matrix of partial derivatives of the component functions at x:

30 J f ( x) := D f ( x) = f 1( x). f m( x) = R m n f 1 x 1 ( x). f m ( x) x 1 f 1 ( x) x n. f m ( x) x n

31 Theorem ( Multivariate Chain Rule) Let X R n suppose g : X Y, where Y R m. Further, suppose f : Y Z, where Z R p. If x is an interior point of X, g( x) an interior point of Y, and g and f are differentiable at x and g( x), respectively, then f g is differentiable at x and: D[f g](x) = Df (g( x))dg( x)

32 Definition (Hessian matrix) Let X R n be open, suppose f : X R, and let x be an element of X. If all second order partial derivatives of f are defined at x, then, in the same way as the first order partial derivatives could be gathered in a vector the gradient, all the second order partial derivatives can be gathered in a matrix. Such a matrix, denoted H f ( x), is called the Hessian of f at x, is square, and should be thought of as a generalized second order derivative for multivariate real valued functions. f1 ( x) f 1,1 ( x) f 1,2 ( x) f 1,n ( x) H f ( x) = f2 ( x). = f 2,1 ( x) f 2,2 ( x) f 2,n ( x) fn ( x) f n,1 ( x) f n,2 ( x) f n,n ( x)

33 Definition (Function of class C k ) Let X R n be an open set, Y R, and suppose f : X Y. f is said to be of class C k on X, denoted f C k (X, Y ) 2, if all partial derivatives of order less or equal to k exist and are continuous on X.

34 Theorem ( Schwarz s Theorem / Young s Theorem) If f C k (X ), then the order in which the derivatives up to order k are taken can be permuted.

35 Theorem ( nth Order Univariate Taylor Approximation) Let X R be an open set and consider f C n+1 (X ). Then f can be best nth order approximated around x by the nth order Taylor expansion: f ( x + h) f ( x) + n f (k) ( x)h k k=1 where h R is such that x + h X and f (k) (x) denotes f s derivative of order k at x. The error of approximation, also known as the remainder of the Taylor approximation, is given by the following formula: k! R n (h x) := f ( x +h) f ( x) n f (k) ( x)h k k! = f (n+1) (x + λh) h n+1 (n + 1)!

36 Theorem ( Second Order Multivariate Taylor Approximation) Let X R n be an open set, f C 3 (X ). Then f can be best 2nd order approximated around x by the second order Taylor expansion: f ( x + h) f ( x) + f ( x) h h H f ( x)h where h R n is such that x + h X. As h approaches zero, the remainder approaches zero at a faster rate than h itself. R 2 (h x) := f ( x + h) f ( x) f ( x) h h H f ( x)h = for some λ (0, 1). f (n) (x + λh) h n+1 (n + 1)!

37 Theorem ( Multivariate Convexity ) Let X be a convex subset of R n. A real-valued function f : X R that is also an element of C 2 (X ) is convex if and only if, H f (x) is positive semidefinite for all x Int(X ). Further, if H f (x) is positive definite for all x Int(X ), then f is strictly convex.

38 Contents Multivariate functions Invertibility Convexity, Concavity, and Multivariate Real-Valued Functions Multivariate differential calculus Introduction Revision: single-variable differential calculus Partial Derivatives and the Gradient Differentiability of real-valued functions Differentiability of vector-valued functions Higher Order Partial Derivatives and the Taylor Approximation Theorems Characterization of convex functions Integral theory Univariate integral calculus The definite integral and the fundamental theorem of calculus Multivariate extension

39 Theorem ( Calculation rules for indefinite integrals) Let f, g be two integrable functions 3 and let a, C be constants, n N. Then (af (x) + g(x))dx = a f (x)dx + g(x)dx x n dx = x n C if n 1 and dx = lnx + C n + 1 x e x dx = e x + C and e f (x) f (x)dx = e f (x) + C (f (x)) n f (x)dx = 1 n + 1 (f (x))n+1 + C if n f (x) 1 and f dx = lnf (x) + C (x)

40 Theorem ( Integration by parts) Let u, v be two differentiable functions. Then, u(x)v (x)dx = u(x)v(x) u (x)v(x)dx.

41 Theorem Let f : [a, b] R be continuous and define F (x) = all x [a, b]. Then, F is differentiable on (a, b) with F (x) = f (x) for all x (a, b). x a f (t)dt for

42 Theorem ( Fubini s theorem) Let I x = [a, b] and I y = [c, d] be two intervals in R n and define I := I x I y. Let f : I R continuous. Then ( ) f (x, y)d(x, y) = f (x, y)dy dx, I I y and all the integrals on the right-hand side are well-defined. I x

43 Theorem Let A, B R two closed intervals, f : A R, g : B R continuous functions. Then A B ( f (x)g(y)d(x, y) = f (x)dx A )( B ) g(y)dy.