Lecture 7: Interchange of integration and limit

Transcription

1 Lecture 7: Interchange of integration an limit Differentiating uner an integral sign To stuy the properties of a chf, we nee some technical result. When can we switch the ifferentiation an integration? If the range of the integral is finite, this switch is usually vali. Theorem (Leibnitz s rule) If f (x,θ), a(θ), an b(θ) are ifferentiable with respect to θ, then θ b(θ) a(θ) f (x,θ)x = f (b(θ),θ)b (θ) f (a(θ),θ)a (θ)+ In particular, if a(θ) = a an b(θ) = b are constants, then θ b a f (x,θ)x = b a f (x,θ) x b(θ) a(θ) If the range of integration is infinite, problems can arise. f (x,θ) x Whether ifferentiation an integration can be switche is the same as whether limit an integration can be interchange. UW-Maison (Statistics) Stat 609 Lecture / 15

2 Interchange of integration an limit Note that f (x,θ) x = lim δ 0 f (x,θ + δ) f (x,θ) x δ Hence, the interchange of ifferentiation an integration means whether this is equal to θ f (x,θ)x = lim δ 0 f (x,θ + δ) f (x,θ) x δ An example of invali interchanging integration an limit Consier { δ 1 0 < x < δ f (x,δ) = 0 otherwise Then, lim δ 0 f (x,δ) = 0 for any x an, hence, 0 = lim δ 0 δ f (x,δ)x lim f (x,δ)x = lim δ 0 δ 0 0 δ 1 x = 1 UW-Maison (Statistics) Stat 609 Lecture / 15

3 We now give some sufficient conitions uner which interchanging integration an limit (or ifferentiation) is vali. The proof of the main result is technical an out of the scope of this course. Theorem (Lebesgue s ominate convergence theorem) Suppose that the function h(x,y) is continuous at y 0 for each x, an there exists a function g(x) satisfying (i) h(x,y) g(x) for all x an y in a neighborhoo of y 0 ; (ii) g(x)x <. Then, lim h(x,y)x = lim h(x,y)x y y 0 y y 0 The key conition in this theorem is the existence of a ominating function g(x) with a finite integral. Applying Theorem to the ifference [f (x,θ + δ) f (x,θ)]/δ, we obtain the next result. UW-Maison (Statistics) Stat 609 Lecture / 15

4 Theorem Suppose that f (x,θ) is ifferentiable at θ = θ 0, i.e., f (x,θ 0 + δ) f (x,θ 0 ) f (x,θ) lim = δ 0 δ θ=θ0 exists for every x, an there exists a function g(x,θ 0 ) an a constant δ 0 > 0 such that (i) f (x,θ 0+δ) f (x,θ 0 ) δ g(x,θ 0 ) for all x an δ δ 0 ; (ii) g(x,θ 0)x <. Then f (x,θ)x f (x,θ) θ = θ=θ0 x θ=θ0 This result is for θ at a particular value, θ 0. Typically f (x,θ) is ifferentiable at all θ, an f (x,θ 0 + δ) f (x,θ 0 ) δ = f (x,θ) θ=θ0 +δ (x) for some δ (x) with δ (x) δ 0, which leas to the next result. UW-Maison (Statistics) Stat 609 Lecture / 15

5 Corollary Suppose that f (x,θ) is ifferentiable in θ an there exists a function g(x,θ) such that (i) f (x,ϑ) ϑ g(x,θ) for all x an ϑ such that ϑ θ δ0 ; (ii) g(x,θ)x < for all θ. Then f (x,θ) f (x,θ)x = x θ Example Let X have the exponential pf f (x) = λ 1 e x/λ, x > 0 where λ > 0 is a constant. We want to prove a recursion relation E(X n+1 ) = λe(x n ) + λ λ E(X n ), n = 1,2,... Note that both E(X n ) an E(X n+1 ) are functions of λ. UW-Maison (Statistics) Stat 609 Lecture / 15

6 If we coul move the ifferentiation insie the integral, we woul have λ E(X n ) = x n ( ) x n λ 0 λ e x/λ x = 0 λ λ e x/λ x x n ( x ) = λ 2 λ 1 e x/λ x = 1 λ 2 E(X n+1 ) 1 λ E(X n ) 0 which is the result we want to show. To justify the interchange of integration an ifferentiation, we take g(x,λ) = x n e x/(λ+δ ( ) 0) x (λ δ 0 ) λ δ 0 Then ( x n ξ ξ e x/ξ ) x n e x/ξ x ξ + 1 g(x,λ), ξ λ δ 0 ξ 2 an we can apply Corollary In the proof of Theorem (ifferentiating mgf to obtain moments), we interchange ifferentiation an integration without justification. We now provie a proof. UW-Maison (Statistics) Stat 609 Lecture / 15

7 Completion of the proof of Theorem Assuming that M X (t) exists for t [ h,h], h > 0, we want to show t M X (t) = t It is enough to show (why?) e tx f X (x)x = e tx f X (x)x = t 0 For t ( h/2,h/2) an x > 0, t etx f X (x) = xe tx f X (x) 0 t etx f X (x)x t etx f X (x)x xehx/2 f X (x) = g(x) For sufficiently large x > 0, x e hx/2 an, hence, 0 g(x)x 0 e hx f X (x)x = M X (h) < An application of Corollary proves the result. We now complete the proof of Theorem C1 an establish another useful result for inverting a characteristic function. UW-Maison (Statistics) Stat 609 Lecture / 15

8 Completion of the proof of Theorem C1. We give a proof for the case where X has pf f X (x). If we can exchange the ifferentiation an integration, then r φ X (t) t r = r e ıtx r t r f X (x)x = e ıtx t r f X (x)x = ı r x r e ıtx f X (x)x an the result follows by setting t = 0 at the beginning an en of the above expression. The exchange is justifie by the ominate convergence theorem, since r e ıtx t r f X (x) ı = r x r e ıtx f X (x) x r f X (x) an E X r = x r f X (x)x < Theorem C7. Let φ(t) be a chf satisfying φ(t) t <. Then the corresponing cf F has erivative F (x) = 1 e ıtx φ(t)t 2π UW-Maison (Statistics) Stat 609 Lecture / 15

9 Proof. Let x δ an x + δ be continuity points of F. By Theorem C2, T e ıt(x δ) e ıt(x+δ) F(x + δ) F(x δ) = lim φ(t)t T T 2π ıt T sin(tδ)e ıtx = lim φ(t)t T T πt sin(tδ)e ıtx = φ(t)t πt where the last equality follows from the fact that sin(tδ)e ıtx φ(t) πtδ φ(t) which is integrable. Also, from the previous boun we can apply the ominate convergence theorem to interchange the integration an limit in the following operation to finish the proof: UW-Maison (Statistics) Stat 609 Lecture / 15

10 Example F F(x + δ) F(x δ) (x) = lim δ 0 2δ sin(tδ)e ıtx = lim φ(t)t δ 0 2πtδ sin(tδ)e ıtx = lim φ(t)t δ 0 2πtδ = 1 e ıtx φ(t)t 2π We now apply Theorem C7 to obtain the pf corresponing to the chf φ(t) = e t : F (x) = 1 e ıtx e t t = 1 cos(tx)e t t 2π π 0 Since e at cos(bt)t = e at [acos(bt) + b sin(bt)]/(a 2 + b 2 ), F (x) = 1 e t [ cos(xt) + x sin(xt)] t= 1 π 1 + x 2 = π(1 + x 2 ) t=0 UW-Maison (Statistics) Stat 609 Lecture / 15

11 We now turn to the question of when we can interchange ifferentiation an infinite summation. Theorem Suppose that the series h(θ,x) converges for all θ in an interval (a,b) R an h(θ,x) is continuous in θ for each x; (ii) h(θ,x) converges uniformly on every close boune subinterval of (a, b). Then (i) θ h(θ,x) = h(θ,x) This theorem is from calculus (or mathematical analysis) an is not easy to apply because we nee to show the uniform convergence of an infinite series. It is more convenient to apply the following ominate convergence theorem for infinite sums. UW-Maison (Statistics) Stat 609 Lecture / 15

12 Theorem 2.4.2A (ominate convergence theorem) Suppose that the function h(x,y) is continuous at y 0 for each x, an there exists a function g(x) satisfying (i) h(x,y) g(x) for all x an y in a neighborhoo of y 0 ; (ii) x g(x) <. Then, lim y y 0 h(x,y) = lim h(x,y) y y x x 0 If h(x,θ) is ifferentiable in θ an there is a function g(x,θ) such that g(x,θ) for all x an ϑ such that ϑ θ δ 0 ; (i) h(x,ϑ) ϑ (ii) x g(x,θ) < for all θ. Then θ x Example h(x,θ) h(x,θ) = x Let X be a iscrete ranom variable having pmf f X (x) = P(X = x) = θ(1 θ) x, x = 0,1,2,... where θ (0,1) is a fixe constant. We want to erive E(X). UW-Maison (Statistics) Stat 609 Lecture / 15

13 Note that θ(1 θ) x = 1 If interchange of summation an ifferentiation is justifie, 0 = θ = = 1 θ Thus, E(X) = (1 θ)/θ. θ(1 θ) x = [(1 θ) x θx(1 θ) x 1 ] θ(1 θ) x 1 1 θ = 1 θ 1 1 θ E(X) θ(1 θ)x xθ(1 θ) x To justify the interchange, let h(x,θ) = θ(1 θ) x an then h(θ,x) = (1 θ)x θx(1 θ) x 1 Consier θ [c,] (0,1). UW-Maison (Statistics) Stat 609 Lecture / 15

14 (1 θ) x θx(1 θ) x 1 (1 c) x + x(1 c) x 1 = g(x) Since [(1 c) x + x(1 c) x 1 ] < the exchange of sum an ifferentiation is justifie. Monotone convergence theorem In some cases, the following result is more convenient to apply. Suppose that functions g n (x), n = 1,2,..., x R, satisfying that 0 g n (x) g n+1 (x) for all n an x an lim n g n (x) = g(x) for all x. Then lim g n (x)x = lim g n(x)x = g(x)x n n which hols even if g(x) is not integrable (i.e., both sies are infinity). The same result hols when is replace by for iscrete x. Proof. We prove the continuous case, since the iscrete case is similar. If g(x)x <, then this result is a irect consequence of the ominate convergence theorem. UW-Maison (Statistics) Stat 609 Lecture / 15

15 Suppose now that g(x)x =. Then g(x)i { x M,g(x) M} x = lim M where I A is the inicator function of A. For each M > 0, g(x)i { x M,g(x) M} x 2M 2 < an g n (x)i { x M,g(x) M} is monotone an converges to g(x)i { x M,g(x) M}. Then, for every M. lim n g n (x)x lim = Since the right sie goes to, n lim n g n (x)i { x M,g(x) M} x g(x)i { x M,g(x) M} x g n (x)x = UW-Maison (Statistics) Stat 609 Lecture / 15