On-Line Appendix. Matching on the Estimated Propensity Score (Abadie and Imbens, 2015)

Transcription

1 On-Line Aendix Matching on the Estimated Proensity Score Abadie and Imbens, 205 Alberto Abadie and Guido W. Imbens Current version: August 0, 205 The first art of this aendix contains additional roofs. The second art reorts the results of a Monte Carlo study that confirms the theoretical roerties of the roensity score matching estimators derived in the article. I. Additional Proofs: We first state and rove a number of reliminary results. For real numbers a, a is the largest integer less than or equal to a, and a is the smallest integer greater than or equal to a. If a is an integer, then a = a, otherwise a = a +. Lemma A. Consider two indeendent samles of sizes n 0 and n from continuous distributions F 0 and F with common suort: X 0,,..., X 0,n0 i.i.d. F 0 and X,,..., X,n i.i.d. F. Let = n 0 + n. Assume that the suort of F 0 and F is an interval inside 0,. Let f 0 and f be the densities of F 0 and F, resectively. Suose that for any x in the suorts of F 0 and F, f x/f 0 x r. For i n and m M n 0, let U n0,n,i m be the m-th order statistic of { X,i X 0,,..., X,i X 0,n0 }. Then, for n 0 3: E n M i= M U n0,n,i m r m= n /2 n 3/4 0 + M n 0 ex n /4 0. /2 nm /4 Proof of Lemma A.: Consider balls assigned at random among n bins of equal robability. It is known that the mean of the number of bins with exactly m balls is equal to m n m m n n see Johnson and Kotz, 977. Because f x/f 0 x r, for any measurable set A: f x PrX,i A = f x dx = f 0 x dx r PrX 0,i A. f 0 x A A Divide the suort of F 0 and F in n 3/4 0 cells of equal robability, / n 3/4 0, under F 0. Let Z M,n0 be the number of such cells that are not occuied by at least M observations from the samle: X 0,,..., X 0,n0. Let µ M,n0 = EZ M,n0. otice that n 0 3 imlies n 3/ Then, m µ M,n0 = M m=0 M m=0 n 3/4 0 n0 n 3/4 0 nm 0 m! Mn M /4 0 m n 3/4 0 n 3/4 0 n 3/4 0 m n0. n 3/4 0 n 3/4 0 n0 m n0 m

2 Using Markov s inequality, PrZ M,n0 > 0 = PrZ M,n0 µ M,n0 Mn M /4 0 n0. n 3/4 0 otice that for any ositive a, we have that a loga. Therefore, for any b <, we have that log b/ b/ and b/ ex b. As a result, we obtain: n0 n0 = n/4 0 ex n /4 n 3/4 0. n 0 0 Putting together the last two dislayed equations, we obtain the following exonential bound for PrZ M,n0 > 0: PrZ M,n0 > 0 Mn M /4 0 ex n /4 0. otice that U n0,n,i m. For 0 n n 3/4 0, let c n0,n be the oint in the suort of F 0 such that c n0,n = F0 n/ n 3/4 0, then M E U n0,n,i m ZM,n0 = 0 m= ow, E n i= M M U n0,n,i m m= = E + E + = + n 3/4 0 n= M r n 3/4 0 M r n 3/4 0. E M c n0,n c n0,n Pr cn0,n X,i c n0,n ZM,n0 = 0 n 3/4 0 n= n M i= n M i= n M n i= cn0,n c n0,n M ZM,n0 U n0,n,i m = 0 PrZ M,n0 = 0 m= M ZM,n0 U n0,n,i m > 0 PrZ M,n0 > 0 m= M ZM,n0 U n0,n,i m = 0 m= PrZ /2 M,n 0 > 0 n M E U n0,n M,i m Z M,n0 = 0 n i= m= PrZ /2 M,n 0 > 0 n r /2 n 3/4 0 + M n 0 ex n /4 0. /2 nm /4 Lemma A.2 Suose that the roensity score X = PrW = X is continuously distributed, with continuous density, f, and interval suort,,, with > 0 and <. Let f w be 2

3 the density of the roensity score conditional on W = w, where w {0, }. Then, the densities f and f 0 are continuous and share a common suort. Moreover, the ratio f /f 0 is continuous, uniformly bounded, and uniformly bounded away from zero for all such that f > 0. Proof of Lemma A.2: Alying Bayes Theorem, for all 0, f PrW = w X = f w = PrW = w = PrW = w f. Therefore, the functions f and f 0 are continuous, and the suorts of f and f 0 are equal to the suort of f. ow, for any such that f > 0, we obtain, Similarly, f f 0 = PrW = 0 PrW = < f 0 f <. Therefore, we obtain η = / >, and η < f f 0 < η.. Lemma A.3 Inverse Moments of the Doubly Truncated Binomial Distribution Let 0 be a Binomial variable with arameters, that is left-truncated for values smaller than M and right-truncated for values greater than M, where M < /2. Let = 0. Then, for any r > 0, there exist a constant C r, such that r r E C r and E C r for all > 2M. 0 Proof of Lemma A.3: Here, we rove the first assertion of the lemma. The roof of the second assertion is analogous. Let = 0. Let q be a scalar greater than one. Then, r r { } r { } E = E > q + E q r Pr > q + q r M 0 r = Pr > q + q r. M otice that: Pr > q = x M x> / q,x M x M x x x x x x. x M 3

4 For > 2M the denominator can be bounded away from zero. Therefore, for some ositive constant C, and q > /, Pr > q C x M x> / q,x M C x> / q x x x x x x C ex { 2 / q 2 }, by Hoeffding s Inequality e.g. van der Vaart and Wellner, 996, Therefore E/ 0 r is uniformly bounded for > 2M. For a samle of scalars X,..., X from a cumulative distribution function F : a, b 0,, let F be the emirical cumulative distribution function: F x =,x X i. i= Let F be the emirical inverse cumulative distribution function: { F q = x : F } x q. inf axb Let ξ :,..., ξ : be the order statistics for a random samle of size from the uniform distribution. Let Ĝ and Ĝ be the emirical cumulative distribution function for that samle and its inverse. Lemma A.4 Suose F : a, b 0, is a continuous and strictly increasing cumulative distribution function with F a = 0. Then, i ii su 0q F q F q 0, a.s. max F ξ i: F i/ a.s. 0, i=: iii for any two integers J and M: max i=j+: M F ξ i+m: F ξ i J: a.s. 0. Proof of Lemma A.4: First, notice that F is uniformly continuous, because it is a continuous function defined on a comact set e.g., Rudin 976, theorems 4.7 and 4.9. Because F has the same distribution as F Ĝ and su Ĝ q q a.s. 0 0q see Shorack and Wellner, 2009, age 95, the result in art i is imlied by uniform continuity of F as follows. Fix δ > 0. Because if F is uniformly continuous, then for each δ > 0, there 4

5 exists a ε > 0 such that if q < ε, then F F q < δ. With robability one, for each ε > 0 there exists ε such that for ε, su Ĝ q q < ε. 0q Therefore, with robability one, there exists ε such that for ε, su F Ĝ q F q < δ, 0q which roves i. Part ii is directly imlied by i and by the fact that F has the same distribution as F Ĝ. To rove iii notice that max F ξ i+m: F ξ i J: max F ξ i+m: F i + M/ i=j+: M i=j+: M + max F ξ i J: F i J/ i=j+: M + max F i + M/ F i J/. i=j+: M Result ii imlies that the first two terms on the right-hand-side of last equation converge almost surely to zero. Then, uniform continuity of F imlies that the last term on the right-hand-side of last equation converges to zero, which roves iii. Lemma A.5 Let Y,... Y + be indeendent and distributed as standard exonential equivalently, Γ,, where Γ denotes the Gamma distribution with arameters,. Let S j = j i= Y i and S +,j = j i= Y i/ + i= Y i, for j +. Let k denote a ositive integer. Then: i S j has a Gamma distribution with arameters j, and moments ES j k = j + k! j! iii S +,j has a Beta distribution with arameters j, j + and moments: ES +,j k = j + k!! j! + k!. Proof of Lemma A.5: See, e.g., Poirier 995, ages Lemma A.6 Suose F : a, b 0, is a strictly increasing and absolutely continuous cumulative distribution function with F a = 0 and derivative fx. Suose m : a, b R is non-negative and continuous hence, bounded on a, b. Then, for any nonnegative integer, M: M i=m+ m F ξ i: ξ i+m: ξ i M: 2M b a msfsds, and M i=m+ m F ξ i: ξ i+m: ξ i M: 2 2M2M + b a msfsds. 5

6 Proof of Lemma A.6: We first rove three results, M m F ξ i: m F i/ ξ i+m: ξ i M: = o, A. i=m+ and M i=m+ m F i/ ξ i+m: ξ i M: 2M M i=m+ m F i/ which together imly the first result in the lemma. b a M i=m+ m F i/ = o, msfsds = o, A.2 A.3 First consider A.. For large enough, we have M + M. Let k be an integer such that M k M. It is enough to rove M m F ξ i: m F i/ ξ i+k+: ξ i+k: = o. i=m+ Then, equation A. follows from summation from k = M to k = M. Because m is uniformly continuous on a, b, then for any ε > 0, there exists a δ > 0 such that if x and x 2 belong to a, b and x 2 x < δ, then mx 2 mx < ε. Consider ε > 0. Then, there exists a δ > 0 such that, alying Lemma A.4ii we obtain: Pr max m F ξ i: m F i/ < ε i=,..., Pr max F ξ i: F i/ < δ. i=,..., Because this derivation holds for any ε > 0, we obtain: max m F ξ i: m F i/ 0. i=,..., A.4 Therefore M i=m+ m F ξ i: m F i/ ξ i+k+: ξ i+k: < max m F ξ i: m F i/ M ξ i+k+: ξ i+k: i=,..., i=m+ which roves A.. ow consider A.2. As before, it is enough to rove: M i=m+ m F i/ ξ i+k+: ξ i+k: M i=m+ m F i/ = o, 0, for M k M. For i =,..., +, define the sacings δ i = ξ i: ξ i : ξ 0: = 0 and ξ +: =. The left-hand-side of last equation can be exressed as: M i=m+ m F i/ δ i+k+. with 6

7 We will show that the last equation converges to zero in robability by showing that its mean and variance converge to zero. Using Lemma A.5, the reresentation of uniform sacings via exonential variables see, e.g., Shorack and Wellner, 2009, age 72, we obtain: Eδ i+k+ = +, varδ i+k+ = < 2. otice also that for any k, l {,, } with k < l, x 0, and y 0, x, we have: l Pr k δ j y δ j = x = Prξ l ξ k y ξ k = x. j=k+ j= A.5 The distribution of ξ l ξ k conditional on ξ k = x is the same as the distribution of l k-th order statistic in a samle of size k from a uniform distribution on 0, x. As a result, the robability of the event ξ l ξ k y conditional on ξ k = x is non-decreasing in x. Therefore, l j=k+ δ j is negatively regression deendent on k j= δ j see, e.g., Lehmann, 966. By exchangeability, this result extends to any two sums S, Q of disjoint sets of uniform sacings. Then, negative regression deendence imlies that for any non-decreasing function, e: coves, eq < 0. A.6 Because m is bounded on a, b, M E m F i/ δ i+k+: = + i=m+ Because m is non-negative, M var m F i/ δ i+k+: i=m+ Therefore A.2 holds. Finally, consider A.3: lim ext we will rove, M i=m+ m F i/ = 0 M i=m+ 2 M i=m+ m F i/ 0. m F i/ 2 var δ i+k+: M i=m+ mf ydy = m F i/ 2 0. b a msfsds. M m F ξ i: m F i/ ξ i+m: ξ i M: 2 = o, A.7 M i=m+ i=m+ m F i/ ξ i+m: ξ i M: 2 2M2M + M i=m+ m F i/ = o, A.8 which along with A.3 imly the second result in the Lemma. Because of equation A.4, to rove equation A.7 it is sufficient to rove that M i=m+ ξ i+m: ξ i M: 2 7

8 is bounded in robability. By Lemma A.5, for 2M: M E ξ i+m: ξ i M: 2 2M2M + = 2M 2M2M i=m+ Also, because of equation A.6, exchangeability, and the Cauchy-Schwarz inequality: var M ξ i+m: ξ i M: 2 i=m+ M = var + M i=m+ M i=m+ M i=m+ i=m+ + 4M 2 δ i+m + + δ i M+ 2 var δ i+m + + δ i M+ 2 + i+2m 0 i 2M + j i cov δ i+m + + δ i M+ 2, δ j+m + + δ j M+ 2 var δ i M+ + + δ i+m 2 M i=m+ var δ i M+ + + δ i+m 2. Lemma A.5 imlies: var δ i M+ + + δ i+m 2 < 2 E δ i M+ + + δ i+m 4 Therefore, var M i=m+ = 2 2M + 3!! 2M! + 4!. ξ i+m: ξ i M: 2 0, which roves A.7. Using Lemma A.5, notice that the exectation of left hand side of equation A.8 is: M i=m+ E m F i/ ξ i+m: ξ i M: 2 = 2M2M + M 2M2M + 2M2M + m F i/ i=m+ A.9 Using equation A.6, Cauchy-Schwarz inequality, exchangeability of the uniform sacings, and 8

9 boundedness of m, we obtain that for some constant C m : M var m F i/ ξ i+m: ξ i M: 2 i=m+ C 2 m + M i=m+ C 2 m M i=m+ var δ i+m + + δ i M+ 2 m + i+2m F i/ M i=m+ + 4M 2C 2 m which roves A.8. 0 i 2M + j i m F j/ cov δ i+m + + δ i M+ 2, δ j+m + + δ j M+ 2 var δ i M+ + + δ i+m 2 M i=m+ var δ i M+ + + δ i+m 2 0, Lemma A.7 Let X be a scalar random variable with interval suort a, b, distribution function G, and density function g that is continuous on a, b. For a random samle X,..., X, let X j: be the j-th order statistic. We will adot the convention X j: = a if j < and X j: = b if j >. Let V k be the rank of observation k in the samle. Let P k be the robability that observation k will be a match for an out-of samle observation with continuous density f: P k = + + XVk : +X Vk +M: /2 a XVk : +X Vk +M: /2 X Vk : +X Vk M: /2 b X Vk : +X Vk M: /2 fx dx {V k M} fx dx {M + V k M} fx dx { M + V k }. A.0 Assume that f/g is bounded on a, b. Let σ 2 be a bounded function with domain a, b. Then, and k= σ 2 X k σ 2 X k Pk 2 k= P k fx k gx k fxk gx k GX Vk +M: GX Vk M: 2 2 GX Vk +M: GX Vk M: 2 0, 0. A. A.2 Proof of Lemma A.7: Fix k {, 2,..., } e.g., k =. otice that, P k XVk : +X Vk +M: /2 X Vk : +X Vk M: /2 fx dx = o. Let Z k = σ2 X k P k fx k gx k GX Vk +M: GX Vk M:. 2 9

10 Given that f and g are continuous, there are mean values X f,k,,m and X g,k,,m in X Vk : + X Vk M:/2, X Vk : + X Vk +M: /2 and X Vk M:, X Vk +M:, resectively such that: 0 = σ 2 X k P k f X f,k,,m GX Vk +M: GX Vk M: + o = Z k + σ2 X k g X g,k,,m fxk gx k f X f,k,,m g X g,k,,m = Z k + o O + o. 2 GXVk +M: GX Vk M: + o 2 The last equality follows from equation A.5 and the continuous maing theorem. Therefore Z k 0. ext, we will show that E Z k is bounded uniformly in. Let P k b, P k m, and Pk t be the three terms on the right hand side of equation A.0, arranged in the same order as in the equation. By Cauchy s generalization of the Mean Value Theorem, there is X,M in 0, X 2M:, such that: P b k F X 2M: = F X 2M: GX 2M: GX 2M: f X,M g X,M GX 2M:. Because f/g is bounded and because for any ositive integer r, E GX 2M: r = r 2M + r!! 2M! + r! 2M + r! <, 2M! we obtain that E Pk b r is bounded by a constant that does not deend on or k. Similarly, by Cauchy s generalization of the Mean Value Theorem, there is X k,,m in X Vk M:, X Vk +M:, such that: P m k XVk : +X Vk +M: /2 X Vk : +X Vk M: /2 XVk +M: X Vk M: fx dx fx dx = F X V k +M: F X Vk M: GX Vk +M: GX Vk M: GX V k +M: GX Vk M: = f X k,,m g X k,,m GX V k +M: GX Vk M:. Because f/g is bounded and because for any ositive integer r, E GX Vk +M: GX Vk M: r r 2M + r!! 2M! + r! 2M + r! <, 2M! we obtain that E Pk m r is bounded by a constant that does not deend on. Using an analogous argument it can be shown that for any ositive integer, r, E Pk t r is bounded by a constant that does not deend on. As a result, E Pk m r is bounded by a constant that does not deend on. Because σ 2 is bounded, alying Minkowski inequality we obtain that, for r > 0, E Z k r is uniformly bounded in. This, in combination with the fact that Z k 0, imlies E Z k 0. ow, exchangeability of Z k for any given imlies that 0

11 E Z = E Z = = E Z. Using Markov s Inequality: 2 ε Pr Z k k= > ε E k= Z k k= E Z k = E Z 0, which roves equation A.. The roof of equation A.2 is analogous. Lemma A.8 Assume that i W, X,..., W, X are indeendent and identically distributed, ii X has a continuous distribution, and W has a binary distribution with PrW = 0, ; iii for w = 0,, the cumulative distribution functions of X given W = w, F w, is absolutely continuous with derivative f w ; iv there exists a constant C h such that for h = f /f 0 we have /C h h C h. Let M be a ositive integer not greater than the minimum of = i= W i and 0 =. Let K M i be the number of times observation i is used as a match when each unit is matched with relacement to the closest M units in the oosite treatment grou, and k be any ositive integer. Then, EK M i k W i = w is uniformly bounded in by a finite constant. Proof of Lemma A.8: Let V i denote the rank of X i among the Wi units with W j = W i, so that V i {,..., max 0, }. Also, let X w,j be the j-th order statistic of X-values among the w units with W i = w. Let X w be the w vector with the stacked covariate values for units with W i = w. For v w, define the robability that unit i is a match for unit j, conditional on W i = w, W j = w, X w, V i = v, 0 = n 0 and = n : M, n 0, n,w, v, X w = Pri J M j 0 = n 0, = n, W i = w, W j = w, V i = v, X w. First we rove that for all n 0 M, n M, w {0, }, and v {,..., n w }, E n w M, n 0, n, w, v, X w k < Ch k 2M + k!, A.3 2M! where the exectation on the left-hand-side integrates over the distribution of X w, which has length equal to n w. Let us first focus on the case with w = and M + v n M. Then X,v M + X,v M, n 0, n,, v, X w = Pr X j X,v+M + X,v 2 2 X, W j = 0 Pr X,v M X j X,v+M X, W j = 0 = F 0 X,v+M X F0 X,v M X. By continuity of F and F 0, boundedness of h, Cauchy s version of the Mean Value Theorem,

12 exchangeability of uniform sacings, and Lemma A.5 we obtain: n En M, n 0, n,, v, X k k E F0 X,v+M F0 X,v M = n k F0 X,v+M F0 X,v M E k F X,v+M F X,v M F X,v+M F X,v M Chn k k E ξ v+m:n ξ v M:n k = n k Ch k 2M + k! 2M! < Ch k 2M + k!. 2M! n! n + k! For the case with w = and v M we have: M, n 0, n,w, v, X w = Pr X j X,v + X,v+M 2 X, W j = 0 Pr X j X,v+M X, W j = 0 F 0 X,2M X. Using the same argument as before, the exectation of n M, n 0, n,, v, X w k can be bounded by Ch k 2M + k!/2m!. The argument for the case with v + M + is similar and is omitted. This roves A.3. Conditional on X, 0 = n 0, = n, W i =, and V i = v, the random variable K M i follows a Binomial distribution with arameters n 0, M, n 0, n,, v, X. Therefore, E K M i k X, 0 = n 0, = n, W i =, V i = v = k Sk, rn 0!M, n 0, n,, v, X r r=0 k Sk, r r=0 n0 n n 0 r! r n M, n 0, n,, v, X r, A.4 where Sk, r is a Stirling number of the second kind see Johnson, Kotz, and Kem, 993. Taking exectations over X, we obtain: E K M i k 0 = n 0, = n, W i =, V i = v k Sk, r r=0 n0 n r Ch r 2M + r!. A.5 2M! Because the bound does not deend on v, it alies also to EK M i k 0 = n 0, = n, W i =. ow, from Lemma A.3, it follows that: E K M i k W i = k Sk, rc r Ch r 2M + r!. 2M! r=0 The same argument alies to EK M i k W i = 0. Lemma A.9 W, X,..., W, X are indeendent and identically distributed, where X has a continuous distribution on a, b, and W has a binary distribution with PrW = 0,. Let P k be the robability that observation k is used as a match for any articular observation in 2

13 the oosite treatment arm, conditional on W and X Wk. Assume that f /f 0 is bounded and bounded away from zero, then for all δ > 0: max P k = o +δ. k=,..., Proof of Lemma A.9: For the roof, we focus on the case when W k = 0. The derivations for the treated observations are analogous. Let {P 0,,..., P 0,0} be the catchment robabilities for {X 0,,..., X 0,0}. If k is such that k M + and k 0 M, then P 0,k = X0,k+M +X 0,k /2 X 0,k M +X 0,k /2 ow aly the change of variables z = F 0 x to obtain: P 0,k = F0X 0,k+M +X 0,k /2 F 0X 0,k M +X 0,k /2 F0X 0,k+M F 0X 0,k M f xdx. f F0 z f 0 F0 z dz. f F0 z f 0 F0 z dz Using the assumtion that f /f 0 is bounded by some constant C, we obtain: P 0,k CF 0 X 0,k+M F 0 X 0,k M. The derivation k < M + and k > 0 M is similar, so it is omitted. ow the result follows from the fact that the maximal uniform sacing is o +δ for all δ > 0 see, e.g., Theorem in Shorack and Wellner 2009, age 726. Lemma A.0 Assume that the conditions of Lemma A.9 hold. In addition, assume that σ 2 w, x = vary W = w, X = x is uniformly bounded. Then, i and ii w i:w i=w w i:w i=w σ 2 w, X i K M i 2 w σ 2 w, X i K M i w i:w i=w i:w i=w σ 2 w, X i w P i = o, σ 2 w, X i 2 wp 2 i + w P i P i = o. Proof of Lemma A.0: Here we rove ii only. The roof of i is analogous but slightly less involved. In the roof we adot w = 0. The roof for w = is identical after switching treatment subscrits. Re-order the samle units so those units with W i = 0 come first, W = = W 0 = 0. For k 0, l 0, and k l, let µ r,k = EKM r k W, X 0 and µ r,s,k,l = EKM r kks M l W, X 0. We first rove the results for the single match case, with M =. otice that, if M =, then conditional on W and X 0, the vector K,..., K 0 has a multinomial distribution with arameters and P 0,,..., P 0,0. The moment generating function of this distribution is: Mt,..., t 0 = 0 3 P 0,k e k t. k=

14 Then, µ 2,k = P 0,k + P 2 0,k, µ 2,2,k,l = P 0,k P 0,l + 2P 2 0,kP 0,l + P 0,k P 2 0,l + 2 3P 2 0,kP 2 0,l. Therefore, µ 2,2,k,l µ 2,k µ 2,l = 0 P 0,k P 0,l P0,kP 2 0,l + P 0,k P0,l P 2 0,kP 2 0,l 0. ow, E = i= 0 0 = σ 2 0, X i K 2 i µ 2,i i= j= 0 i= 0 i= 2 W, X0 σ 2 0, X i σ 2 0, X j E Ki 2 µ 2,i Kj 2 µ 2,j W, X0 σ 4 0, X i µ 4,i µ 2 2,i σ 4 0, X i µ 4,i. Therefore, from equation A.5, we obtain E 0 0 i= 0 σ 2 0, X i σ 2 0, X j µ 2,2,i,j µ 2,i µ 2,j i= j i 2 σ 2 0, X i Ki 2 W µ 2,i C σ 4 Eµ 4,i W 0 C σ r=0 S4, r 0 r C r hr +! where C σ 4 is a bound on σ 4 w, x for w {0, } and x in the suort of X. ow, Lemma A.3 imlies 0 2 E σ 2 0, X i Ki 2 µ 2,i 0, 0 i= which yields ii for the case of w = 0 and M =. ow consider the case with M >, E = i= 0 i= σ 2 0, X i K 2 M i µ 2,i σ 4 0, X i µ 4,i µ 2 2,i W, X0 0 σ 2 0, X i σ 2 0, X j µ 2,2,i,j µ 2,i µ 2,j. A.6 i= j i The roof that the exectation of the first term on the left-hand-side of equation A.6 converges to zero is the same as for the case of M =. However, if M >, then it is not the case that all elements of the second term on the left-hand-side of equation A.6 are negative. If M >, an 4

15 observation i, with W i = 0 and M + V i = v 0 M is a match for all observations in the oosite treatment arm that have covariate value in the catchment interval X0,v+M + X 0,v A M i =, X 0,v M + X 0,v, 2 2 which overlas with the catchment intervals of other 2M untreated observations. The catchment intervals of an untreated observation with V i M or V i > 0 M overlas with less than 2M catchment intervals of other untreated observations. As a result, relative to the roof for M =, overlaing catchment intervals create terms, µ 2,2,i,j µ 2,i µ 2,j, that are not necessarily negative. The number of such otentially non-negative terms is smaller than 2M 0. Let I ij be an indicator function that takes value equal to one if the catchment intervals of observations i and j overla, and value zero otherwise. Let K M i, j be the number of treated observations with covarite values in A M i A M j. Then, i= j i σ 2 0, X i σ 2 0, X j E KM 2 i µ 2,i KM 2 j µ 2,j W, X C σ i= j i 0 i= j i 0 i= j i σ 2 0, X i σ 2 0, X j E KM 2 ikj 2 W, X 0 µ 2,i µ 2,j I ij σ 2 0, X i σ 2 0, X j E KM 2 ikj 2 W, X 0 I ij E KM 4 i, j W, X 0 I ij. Using the same argument as in Lemma A.8, it can be seen E KM 4 i, j W, I ij = 4 k=0 k c k, 0 for some ositive constants, c k. ow, Lemma A.3 yields the result. Lemma A. Suose that the assumtions of Lemma A.0 hold. Assume also that the density of X is continuous on a, b. Finally, assume that, for w = 0,, σ 2 w, x is continuous on a, b. Let = PrW =, then, for w = 0,, i and ii w i:w i=w w i:w i=w σ 2 w, X i K M i σ 2 w, X i K M i 2 ME + ME M2M + 2 σ 2 2w f w X i w, X i f w X i σ 2 w, X i E σ 2 w, X i W i = w 2w f w X i f w X i W i = w 2w 2 f w X i f w X i W i = w. Proof of Lemma A.: The result follows from lemmas A.6, A.7, and A.0. 5

16 Proof of Proosition : Let and R = D = = i= i= i= µ, X i µ0, X i τ 2W i + K M i Y i µw i, X i, M 2W i µ W i, X i M j J M i µ W i, X j. otice that We will first show that D τ τ = D + R. d 0, σ 2. otice that: where D = 2 k= ξ,k, ξ,k = µ, X k µ0, X k τ for k, and ξ,k = 2W k + K M k Y k µw k, X k M for + k 2. Consider the σ-fields F,k = σ{w,..., W k, X,..., X k } for k and F,k = σ{w,..., W, X,..., X, Y,..., Y k } for + k 2. Then for each, i ξ,j, F,i, i 2 j= is a martingale. To obtain the result of the roosition, we aly the Central Limit Theorem for martingale arrays e.g., Billingsley, 995. The following three conditions are sufficient: and 2 k=+ k= Eξ 2,k F,k 2 k= E ξ,k 2+δ 0 for some δ > 0, A.7 Eξ,k F 2,k E µ, X µ0, X τ 2, E σ 2, X + E σ 2 0, X X + 2M X + 2M X X X X A.8. A.9 6

17 Equation A.7 is the Lyaounov condition which is sufficient for the usual Lindeberg condition to hold. Because the functions µw, are continuous on a comact suort, these functions are also bounded. Let C µ be a bound on max w, µw, for w {0, } and,. We obtain k= E ξ,k 2+δ 2C µ + τ 2+δ δ/2 0. ow, let C σ 2+δ be a bound on E Y i µw i, P X i 2+δ W i, P X i. ow, using the Law of Iterated Exectation and the fact that K M i has bounded moments Lemma A.8, we obtain: + K 2+δ M i 2 k=+ E ξ,k 2+δ C σ 2+δE M 0. δ/2 This roves equation A.7. Equation A.8 is easy to rove because the data are i.i.d. To rove equation A.9 notice that: 2 k=+ Eξ 2,k F,k = i= + K 2 M i σ 2 W i, X i. M ow, Lemma A. imlies the result in equation A.9 after some algebra. To finish the roof of art i, we will show that R 0. We can write R = R,0 + R,, where R,w = µ W i, X i M µ W i, X j. i:w i=w j J M i We rove R, = o. The roof for R,0 = o is analogous, so we omit it. By Markov s Inequality, it is enough to show that E R, 0. Without loss of generality and to simlify notation, we reorder the observations in the samle, so that the observations with W i = come first. Because the function µ has a derivative bounded by C µ, we obtain that for 0 = n 0 with 0 = n 0, M n 0 M and n = n 0 : E R, n 0 = n 0 M i= C µ n M i= j J M i j J M i E µ0, X i µ0, X j 0 = n 0 E X i X j 0 = n 0. ow, lemmas A. and A.2 imly that there is some r > 0 such that: E R, 0 = n 0 C µ r n /2 n 3/4 0 + M n 0 ex n /4 0. /2 nm /4 Because n 3/4 0 / n 3/4 0 and n M+/2 0 ex n /4 0 are bounded for all n 0, there exists some constant C such that: Therefore, E R, E R, 0 = n 0 C /4 n 3/4 n 3/4 0 C 3/4 E M 0 M, /4 0. 7

18 which converges to zero because of the result in Lemma A.3. This comletes the roof of art i. To rove art ii, first notice that τ t, τ t = D t, + R t,, where D t, = + W i µ, X i µ0, X i τ t i= i= W i W i K M i Y i µw i, X i M and R t, = i= W i µ0, X i M µ0, X j. j J M i Because R t, = / R,, / /EX and R, 0, we obtain: Rt, 0. d Therefore, we need to show that D t, 0, σ 2 t. otice that: where for k, and ξ t,,k = ξ t,,k = D t, = 2 k= ξ t,,k, W k µ, X k µ0, X k τ t W k W k K M k Y k µw k, X k M for + k 2. Consider the σ-fields F t,,k = σ{w,..., W, X,..., X k } for k and F t,,k = σ{w,..., W, X,..., X, Y,..., Y k } for + k 2. Then for each, i ξ t,,j, F t,,i, i 2 j= is a martingale. To obtain the result of the roosition, we aly the Martingale Central Limit Theorem. The following three conditions are sufficient: 2 k= E ξ t,,k 2+δ 0 for some δ > 0, A.20 and Eξt,,k F 2 t,,k k= EX 2 E X µ, X µ0, X τ t 2, A.2 2 k=+ Eξ 2 t,,k F t,,k + EX 2 E X σ 2, X EX 2 E σ 2 2 X 0, X X + M X + 2M 2 X. X A.22 8

19 Equation A.20 follows from the same arguments emloyed for equation A.7 and from the fact that the moments of / are bounded Lemma A.3. In articular: E ξ t,,k 2+δ k= k= 2+δ/2 E 2+δ = 2C µ + τ t 2+δ E δ/2 2C µ + τ t 2+δ 2+δ 0. Using the Law of Iterated Exectation and the fact that K M i has bounded moments, and the Cauchy-Schwartz inequality we obtain: 2+δ 2 E C σ 2+δE + K 2+δ M i ξ t,,k 2+δ M 0. δ/2 k=+ It is easy to show that 2 k=+ k=+ 2 E 2 W k Y k µ, X k 2 Ft,,k = 2 W i σ 2, X i i= EX 2 E X σ 2, X. In addition: 2 2 KM k Ft,,k E W k Y k µw k, X k 2 M = 2 2 KM i W i σ 2 0, X i. M i= ow, Lemma A. imlies the result in equation A.22. Proof of Lemma : Assumtion 4i imlies that there exist finite constants C L and C U such that C L x θ C U for any θ intθ and x in the suort of X. Because f is continuous and everywhere ositive, it follows that it is bounded away from zero on the comact interval C L, C U see, e.g., Rudin 976, Theorem 4.6. Similarly F F is bounded and bounded away from zero on C L, C U. Therefore, there exist ositive constants c U and c L such that c L f 2 x θ/f x θ F x θ c U, for any θ intθ and x in the suort of X. Then, for θ intθ and any non-zero v R k, we obtain E X 2 f 2 X θ F X θ F X <, θ and v E X f 2 X θ F X θ F X θ X v c L v EXX v > 0. As a result, I θ is finite and ositive definite for all θ intθ. By Assumtion 4 i and ii, f 2 /F F is continuous and bounded in C L, C U and X is bounded. By the Dominated Convergence Theorem, we obtain that I θ is continuous on intθ. This, along with continuous differentiability of F, imlies that the arametrization θ P θ is regular on intθ see Proosition 2.. in Bickel, et al

20 ow, Proosition 2..2 in Bickel, et al. 998 imlies that, under P θ, Λ θ θ = h θ 2 h I θ h + o. Continuity of I θ imlies the result in equation. Also for regular arametric models, equation 2 is derived in the roof of Proosition 2..2 in Bickel et al Suose that θ is asymtotically linear. That is, under P θ θ θ = I θ θ + o. By contiguity see, e.g., Proosition 2..3 in Bickel et al., 998, we obtain that the revious equation holds also under P θ. Therefore, under P θ, θ θ = h + I θ θ + o. ow, equation 3 follows from equation 5 in Proosition 2..2 in Bickel et al Therefore, if θ is asymtotically linear then equation 3 holds. To rove asymtotic linearity of θ, notice first that, given that F x θ is continuously differentiable and bounded away from zero and one on intθ for all x in the suort of X, and that I θ exists and is continuous, Lemma 7.6 in van der Vaart 998 imlies that {P θ : θ intθ} is differentiable in quadratic mean. Moreover, given that fx θ/f x θ and fx θ/ F x θ are uniformly bounded by a constant for all θ intθ and x in the suort of X, and that I θ is nonsingular, then by Theorem 5.39 in var der Vaart 998 consistency of θ imlies that equation 3 holds. ow, given that EXX is nonsingular, F x θ is continuous in θ for all x in the suort of X and bounded away from zero and one, and Θ is comact, Theorem 2.5 in ewey and McFadden 998 imlies θ θ. In the setting of section III, the following lemma rovides rimitive conditions for Assumtion 5. Lemma A.2 Let X 0 be the first coordinate of X and let X be the sub-vector of the last k coordinates of X. Assume that the distribution of X 0 conditional on X, Y resectively, X, Y 0 admits a density, f X0 X,Y x 0, x, y resectively, f X0 X,Y 0 x 0, x, y 0, with resect to the Borel measure, λ. Assume that f X0 X,Y x 0, x, y and f X0 X,Y 0 x 0, x, y 0 are bounded and continuous functions of x 0. For =, 2,..., let X θ = X 0 θ 0 + X θ. Assume that θ θ such that θ 0 0. Let ry, w, x be a bounded function from R k+2 to R, continuous in the first coordinate of x. Then almost surely. E θ ry, W, X W, F X θ ErY, W, X W, F X θ, Proof of Lemma A.2: Because F is strictly increasing, we have E θ ry, W, X W, F X θ = E θ ry, W, X W, X θ. Therefore, it is enough to rove E θ ry, W, X W, X θ ErY, W, X W, X θ, 20

21 almost surely. otice also that E θ ry, W, X W, F X θ = W E θ ry, W, X W =, F X θ + W E θ ry, W, X W = 0, F X θ. It is, therefore, enough to rove convergence of E θ ry, W, X W =, X θ and E θ ry, W, X W = 0, X θ. We will rove convergence for E θ ry, W, X W =, X θ ; the roof for E θ ry, W, X W = 0, X θ is analogous. Let r y, x = ry,, x. Given that W is indeendent of Y, X conditional on the roensity score, we obtain E θ ry, W, X W =, X θ = E θ r Y, X W =, X θ = E θ r Y, X X θ = Er Y, X X θ. Similarly, ErY, W, X W =, X θ = Er Y, X X θ. Hence, we aim to rove Er Y, X X θ Er Y, X X θ almost surely. More recisely, if we make g θ X = Er Y, X X θ, we aim to rove g θ X g θ X as θ θ, almost surely. Let f Y,X y, x be the density of Y, X with resect to some σ-finite measure, µ. Y, X may include variables that are not continuously distributed. Other densities are denoted analogously. By the change of variables formula, if θ 0 0, f X θ,y,x z, y, x = θ 0 f Y,X 0,X = θ 0 f X 0 X,Y y, z x θ, x θ 0 z x θ, x, y f Y,X y, x is the density of X θ, Y, X with resect to λ µ. Because r y, x = ry,, x, the function r is bounded and continuous in the first coordinate of x. With a slight notational abuse we will write r Y, X 0, X = r Y, X. Then, by continuity and boundedness of r and f X0 X,Y, and the Dominated Convergence Theorem, we obtain that for all x in the suort of X r y, x θ x θ x θ x, x f θ X0 X θ,y, x, y f Y,X y, x dµ 0 θ 0 θ 0 x θ x f θ X0 X,Y, x, y θ 0 f Y,X y, x dµ which is a version of g θ x, converges to r y, x θ x θ x θ x θ0, x f θ X0 X,Y θ0, x, y f Y,X y, x dµ x θ x f θ, X0 X,Y θ0, x, y f Y,X y, x dµ, which is a version of g θ x. 2

22 II. Monte Carlo Evidence: In this section we reort the results of a simulation exercise designed to investigate the samling distribution of roensity score matching estimators and the quality of the aroximation to that distribution that is roosed in this article. The Monte Carlo results in this section illustrate the effect of adjusting standard errors and confidence intervals for the estimation error in the roensity score and confirm our theoretical results. We reort results for five designs and for the four tye of estimators considered in section II. In all cases we use = 5000 observations, two covariates, and 2000 Monte Carlo relications. For each design and each estimand ATE or ATET we calculate two estimators. The first estimator is based on matching on the true roensity score, τ = τ θ for ATE and τ t, = τ t,θ for ATET. The second estimator is based on matching on the estimated roensity score, τ = τ θ for ATE and τ t, = τ t, θ for ATET. We estimate standard errors in three different ways. For estimators that match on the true roensity score, we construct standard errors using the formulas derived in Abadie and Imbens 2006 for the case when matching is done directly on covariates. Those formulas are valid because in this case matching is done using a covariate, which is the true roensity score. For the case when matching is done on the estimated roensity score, we first estimate standard errors without adjusting for estimation of the roensity score. That is, these are the standard errors that are obtained when the estimated roensity score is used for matching and for the estimation of the standard errors, but the fact that the roensity score is estimated is ignored in the calculation of the standard errors. These standard errors corresond to σ/ for ATE and σ t / for ATET, where σ and σ t are given in section IV. For the case of matching on the estimated roensity score we also calculate the standard errors σ adj / and σ adj,t /, which adjust for estimation of the roensity score. For each estimator/standard error air we evaluate the erformance of nominally asymtotic 95 ercent confidence intervals constructed by adding and subtracting.96 times the standard error to the estimator. Design I: Two covariates, X and X 2, are both uniformly distributed on /2, /2 and indeendent of each other. The otential outcomes are generated by Y 0 = 3X 3X 2 + U 0 and Y = 5 + 5X + X 2 + U, and U 0 and U are indeendent standard ormal random variables, indeendent of W, X, X 2. The treatment variable, W, is related to X, X 2 through the roensity score, which is logistic PrW = X = x, X 2 = x 2 = exx + 2x 2 + exx + 2x 2. ATE and ATET estimators use one match M =. Design II: X and X 2 are distributed as in Design I. Potential outcomes are generated by Y 0 = 0X + U 0 and Y = 5 0X + U, where U 0 and U are standard ormal random variables indeendent of each other and of W, X, X 2. The treatment variable, W, is related to X, X 2 through the roensity score, which is logistic PrW = X = x, X 2 = x 2 = ex2x 2 + ex2x 2. In this design, average treatment effect varies widely as a function of X, so τ t θ / θ is large. In addition, in this design each value of the roensity score is associated with a unique value for the covariates, and therefore both c and c t are equal to zero. ATE and ATET estimators use one match M =. 22

23 Design III: In this design we modify Design I by using four matches M = 4 on the estimated or true roensity score rather than a single match. The remainder of the design, including the otential outcome distributions and the assignment mechanism, is identical to that in Design I. Design IV: This design is identical to Design II, excet the estimated roensity score is missecified as: exx 2 + /2 2 + exx 2 + /2 2. otice that this secification is still a valid balancing score Rosenbaum and Rubin, 983 because the log odds ratio is a monotone function of the covariates. Design V: In this design we change the distribution of the covariates. With robability 0.7, X has a uniform distribution on the interval /2, 0, and with robability 0.3 it has a uniform distribution on the interval 0, /2. With robability 0.6, X 2 has a truncated exonential distribution on the interval /2, 0, and with robability 0.4 it has a truncated exonential distribution on the interval 0, /2. Hence the distribution of the covariates is discontinuous at zero. In Table we resent Monte Carlo results for the ATE and in Table 2 we resent Monte Carlo results for the ATET. In Design I the standard deviation of the estimator based on matching on the true roensity score is equal to for ATE and for ATET row. The average the standard error across simulations are for ATE and for ATET, very close to the standard deviations. Asymtotic 95 ercent confidence intervals rovide coverage close to nominal row 3. As redicted by the theoretical results, matching on the estimated roensity score leads to a smaller standard deviation for ATE, 0.057, row 4 than matching on the true roensity score. For this design, the same is true for ATET: estimation of the ATET arameter matching of the estimated roensity score is more recise than matching on the true roensity score. Row 5 reorts average standard errors when the fact that the roensity score was estimated is ignored in the construction of the standard errors. Row 6 reorts average standard errors that adjust for the estimation of the roensity score. Standard errors that do not account for the estimation of the roensity score are severely biased as they aroximate the standard deviation of the estimator for the case when the roensity score is known in row. In contrast, the adjusted standard errors closely aroximate the standard deviation of the estimators that match on the estimated roensity score in row 4. Accordingly, using unadjusted standard errors to construct confidence intervals leads to over-coverage, while confidence intervals constructed using adjusted standard errors roduce coverage rates that are close to nominal. Our theoretical results redict that in Design II both the adjusted and unadjusted standard errors for ATE should erform well. The reason is that c = 0, so adjusting the ATE standard errors is not necessary. Moreover, our theoretical results redict that, in this design, the adjustment to the ATET standard errors for first ste estimation of the roensity score is ositive. The reason is that, in this design, c t = 0 but τ t θ / θ 0, which imlies that matching on the estimated roensity score roduces estimators with higher variance than matching on the true roensity score. Accordingly, confidence intervals for ATET constructed using matching on the estimated roensity score and unadjusted standard errors lead to under-coverage. The simulation results are consistent with these redictions. In Design III with the number of matches equal to four instead of one, the recision of the estimator imroves. There remains a bias in the standard errors that are based on ignoring the 23

24 estimation of the roensity score. In Design IV the roensity score is missecified. However, the missecified roensity score is still a valid balancing score, and thus matching on it continues to remove the bias from all covariates. Coverage rates of the confidence intervals are still close to nominal levels. In the fifth and last design the distributions of the covariates include discontinuity oints. This aears to have little effect on the erformance of the confidence intervals. Additional References Abadie, A. and Imbens, G.W. 2006b Additional Proofs for Large Samle Proerties of Matching Estimators for Average Treatment Effects, web aendix. Johnson,. and Kotz, S. 977 Urn Models and Their Alications, John Wiley & Sons, ew York. Johnson,., Kotz, S., and Kem, A. 993 Univariate Discrete Distributions, John Wiley & Sons, ew York. Lehmann, E.L. 966 Some Concets of Deendence, Annals of Mathematical Statistics, vol. 37, no. 5, Poirier, D.J. 995 Intermediate Statistics and Econometrics: A Comarative Aroach, MIT Press, Cambridge. Rudin, W. 976 Princiles of Mathematical Analysis, McGraw-Hill, ew York. Shorack, G.R. and Wellner, J.A. 986 Emirical Processes with Alications to Statistics. Wiley, ew York. van der Vaart, A.W. and Wellner, J.A. 996 Weak Convergence and Emirical Processes, Sringer-Verlag, ew York. 24

25 Table I Monte Carlo Results for Average Treatment Effect = 5000, umber of Relications = 2000 Design I II III IV V Panel A: Matching on the True Proensity Score PS Standard Deviation Average Standard Error Coverage Rate of 95% Confidence Interval Panel B: Matching on the Estimated PS Standard Deviation Average Standard Error: Ignoring Estimation of the PS Accounting for Estimation of the PS Coverage Rate of 95% Confidence Interval: Ignoring Estimation of PS Accounting for Estimation of the PS

26 Table II Monte Carlo Results for Average Treatment Effect on the Treated = 5000, umber of Relications = 2000 Design I II III IV V Panel A: Matching on the True Proensity Score PS Standard Deviation Average Standard Error Coverage Rate of 95% Confidence Interval Panel B: Matching on the Estimated PS Standard Deviation Average Standard Error: Ignoring Estimation of the PS Accounting for Estimation of the PS Coverage Rate of 95% Confidence Interval: Ignoring Estimation of PS Accounting for Estimation of the PS