Differentiating under an integral sign

Transcription

1 Department of Mathematics Ma 3/13 KC Border Introduction to Probability and Statistics Winter 17 Supplement 4: Differentiating under an integral sign S4.1 Differentiating through an integral In the derivation of Maximum Likelihood Estimators, or the Cramér Rao Lower Bound, we differentiated under an integral sign, and I never told you explicitly when that is allowed. Here is a theorem that gives conditions under which it is permissible. We start with a function g of a vector variable x and a single parameter θ. Let the domain of g be Θ, where is a rectangle in R n and Θ is an interval in R. So Define the function G: Θ R by g : Θ R. G(θ) = In order for this to make sense we assume: g(θ; x) dx. S4.1.1 Assumption Let be a rectangle in R n, let Θ be an open interval in R. Assume that for every θ Θ the function x g(θ; x) is continuous and has a finite integral. That is, ( ) g(θ; x) dx < θ Θ. We would like to show that G (θ) = D 1 g(θ; x) dx, where D 1 g indicates the partial derivative of g with respect to its first argument θ. In order for this to be true, the partial derivative has to exist and be integrable with respect to x. S4.1. Assumption Assume that every x, the function θ g(θ; x) is continuous, and for every θ Θ, the partial derivative D 1 g(θ; x) exists, and is integrable with respect to x. That is, D 1 g(θ; x) dx <. But this is not enough. As you may remember, a partial derivative is a limit of the form lim h ( g(θ + h, x) g(θ, x) ) /h. You may have heard that passing limits through an integral sign requires a boundedness condition (the Lebesgue Dominated Convergence Theorem). It is beyond the scope of this course and this note to go through the details, but let me say that what is required is called a uniform local integrability condition. To get a feeling for what this means, realize that Assumption S4.1. requires that at each θ the function D 1 g(θ; x) is bounded in absolute value by an integrable function of x, namely D 1 g(θ; x) itself. Uniform local integrability requires that for each θ, there is little strip (θ ε, θ + ε) on which D 1 g(θ; x) is bounded uniformly in θ for each x by an integrable function of x. S4 1

2 Ma 3/13 Winter 17 KC Border Differentiating an integral S4 S4.1.3 Assumption Assume that for every θ Θ there is an ε > and a nonnegative function h: R (depending on θ ) such that for all x, sup D 1 g(θ; x) h(x). θ (θ ε,θ +ε) and These assumptions are sufficient. 4 h(x) dx <. S4.1.4 Theorem Let g : Θ R satisfy Assumptions S4.1.1, S4.1., and S Then the function G: Θ R defined by G(θ) = g(θ; x) dx is differentiable on Θ and G (θ) = D 1 g(θ; x) dx. Proof : For a proof, see Theorem 4.5 of Aliprantis and Burkinshaw [1, pp ] and the remarks following it. Assumption S4.1.3 is awkward to use, so a stronger condition that implies it is useful. The next theorem may be easier to apply. It requires to be closed and bounded, and g to have a jointly continuous partial derivative. It is similar to Theorem in Dieudonné [, p. 177]. S4.1.5 Theorem Let be a closed and bounded rectangle in R n, let Θ be an open interval in R, and let g : Θ R be jointly continuous. Assume that the partial derivative D 1 g(θ; x) is jointly continuous on Θ. Then the function G: Θ R defined by G(θ) = g(θ; x) dx is differentiable on Θ and G (θ) = D 1 g(θ; x) dx. Proof : Since g and D 1 g are jointly continuous, they are bounded on every set of the form [θ ε, θ + ε], and so integrable. Therefore the Hypotheses of Theorem S4.1.4 are satisfied. S4. An illustrative (counter)example The following example shows what can go wrong when the uniform local integrability Assumption S4.1.3 is violated. You can find it in Gelbaum and Olmsted [3, Example 9.15, p. 13], but similar examples are well-known. S4..1 Example Let Θ = R and let = [, 1]. Define g : Θ R via θ 3 /x g(θ, x) = x x >, e θ x =. The function g is plotted in Figure We can make weaker assumptions, see, for instance, Aliprantis and Burkinshaw [1, pp ]. The problem is that the weaker assumptions involve measure-theoretic concepts that are beyond the scope of this course. v ::11.58 KC Border

3 Ma 3/13 Winter 17 KC Border Differentiating an integral S4 3 Surface of graph. Contours. Figure 17.. Plots of g(θ; x) = θ3 x e θ /x, (x > ). KC Border v ::11.58

4 Ma 3/13 Winter 17 KC Border Differentiating an integral S4 4 (Notice that for fixed x, the function θ g(θ, x) is continuous at each θ; and for each fixed θ, the function x g(θ, x) is continuous at each x, including x =. (This is because the exponential term goes to zero much faster than polynomial term goes to zero as x.) The function g is not jointly continuous though: Along the curve x = θ we have g(θ, x) = e 1 /θ, which diverges to as θ and diverges to as θ.) Define 1 G(θ) = g(θ; x) dx = θ x e θ /x dx. Consulting a table of integrals if necessary, we find the indefinite integral 1 x e a/x dx = e a/x /a. Thus, letting a = θ we have G(θ) = θe θ (θ R). The function G is plotted in Figure Differentiation yields G (θ) = (1 θ )e θ (θ R). Note that G () = (1 )e = 1. Figure G(θ). Now let s compute D 1 g(θ; x): For x =, g(θ; x) = for all θ, so D 1 g(θ; ) =. For x >, we have D 1 g(θ; x) = 3θ x e θ /x + θ3 x e θ ( 3θ = x x 3 ) e θ /x. /x ( θ/x) v ::11.58 KC Border

5 Ma 3/13 Winter 17 KC Border Differentiating an integral S4 5 So ( 3θ x D 1 g(θ; x) = x )e θ /x x > 3 x =. (Note that for fixed θ, the limit of D 1 g(θ; x) as x is zero, so for each fixed θ, D 1 g(θ; x) is continuous in x. Also, for each fixed x, D 1 g(θ; x) is continuous in θ. But again, along the curve x = θ, we have D 1 g(θ; x) = ( 3θ θ ) e 1 = e 1 /θ which diverges to as θ. Thus D 1 g(θ; x) is not jointly continuous at (, ). See Figures 17.4 and 17.5.) Define the integral It satisfies I() =. I(θ) = 1 D 1 g(θ; x) dx = 1 ( 3θ x x 3 ) e θ /x dx. At θ =, we have G () = (1 )e = 1 = I() = So the conclusion of Theorem S4.1.4 fails. 1 D 1 g(; x) dx, The failure of the theorem is confined to the case θ =. For θ >, I(θ) can be computed as I(θ) = 1 D 1 g(θ; x) dx = 1 ( 3θ x x 3 ) e θ /x dx 1 = 3θ 1 1 x e θ /x dx θ 4 [ ] = 3θ e θ /x [ x=1 θ θ 4 x= = (1 θ )e θ. 1 e θ /x x 3 e θ /x dx ( 1 θ θ x ) ] x=1 x= For θ >, we have So the only problem is right at θ =. G (θ) = (1 θ )e θ = I(θ) = 1 D 1 g(θ; x) dx. Let s check Assumption S4.1.3 for θ =. We need to find an ε > so that θ < ε implies D 1 g(θ; x) h(x) where 1 h(x) dx <. Now for x >, ( 3θ D 1 g(θ; x) = x x 3 ) e θ /x. Looking at points of the form x = θ, we see that h(x) must satisfy h(x) D 1 g( x; x) = ( 3 x x) e 1 = e 1 /x. KC Border v ::11.58

6 Ma 3/13 Winter 17 KC Border Differentiating an integral S4 6 Surface of the log graph. Contours. Figure Plots of the log of D 1 g(θ; x) = ( 3θ x x )e θ /x. 3 v ::11.58 KC Border

7 Ma 3/13 Winter 17 KC Border Differentiating an integral S4 7 Figure Cross-sections of D 1 g(θ; x) = ( 3θ x x )e θ /x. 3 See Figure So for any ε > for all θ < ε, we have x θ implies h(x) e 1 /x. Thus 1 h(x) dx ε h(x) dx e 1 ε 1 dx =. x Thus Assumption S4.1.3 is violated by this example, and the conclusion of the theorem fails. Bibliography [1] C. D. Aliprantis and O. Burkinshaw Principles of real analysis, 3d. ed. San Diego: Academic Press. [] J. Dieudonné Foundations of modern analysis. Number 1-I in Pure and Applied Mathematics. New York: Academic Press. Volume 1 of Treatise on Analysis. [3] B. R. Gelbaum and J. M. Olmsted. 3. Counterexamples in analysis. Minneola, NY: Dover. Slightly corrected reprint of the 1965 printing of the work originally published in 1964 by Holden Day in San Francisco. KC Border v ::11.58

8