Random Assignment of Defendants to ADAs

Transcription

1 Random Assignment of Defendants to ADAs Modeling and Computations Eric Laber JJ Prescott Department of Statistics University of Michigan, Ann Arbor May 31, 2008

2 Motivation Why random assignment matters Reduce bias Correct (causal) inference Check NODA office claims!

3 Motivation Why random assignment matters Reduce bias Correct (causal) inference Check NODA office claims! Testing random assignment Stationary χ 2 test or time independent Monte Carlo methods (e.g. permutation tests, see Abrams) Non-stationary Monte Carlo methods with time dependence

4 Toy Example: Testing independence of assignment by race. DF ADA White Black White Black

5 Toy Example: Testing independence of assignment by race. DF ADA White Black White Black

6 Toy Example: Testing independence of assignment by race. DF ADA White Black White Black

7 Toy Example: Testing independence of assignment by race. DF ADA White Black White Black Using χ 2 test p-value 0

8 Suppose the racial decomposition of ADAs changes with time Racial Decomposition of ADAs Proportion Proportion White Proportion Black Time

9 Further suppose the racial decomposition of defendants changes with time Racial Decomposition of Defendants Proportion Proportion White Proportion Black Time

10 Consider the following random generating mechanism: For every time t = 1, 2,..., 1000 a defendant is randomly selected according to the proportions given at time t in the preceding display At time t we also select an ADA according to the proportions in the preceding display

11 Consider the following random generating mechanism: For every time t = 1, 2,..., 1000 a defendant is randomly selected according to the proportions given at time t in the preceding display At time t we also select an ADA according to the proportions in the preceding display What would the expected counts be?

12 Expected counts! DF ADA White Black White Black

13 Failure to account for non-stationary population proportions can lead to incorrect inference!

14 Not just a toy example: Racial Decomposition of Trial ADAs Over Time Proportion White ADAs Date (YYYYMMDD) Proportion Black ADAs Date (YYYYMMDD)

15 Not just a toy example: Racial Decomposition of DFDNs at Trial Over Time Proportion White DFDNS Date (YYYYMMDD) Proportion Black DFDNs Date (YYYYMMDD)

16 Accounting for non-stationary distributions Solution: use time-dependent Monte Carlo methods to account for evolving populations

17 Accounting for non-stationary distributions Notation Let t = 1, 2,..., T denote arrive times of cases to the NODA office Let A t be the race* of the screening ADA handling case arriving at time t Let D t be the race* of the defendant at time t Let p t = (B t, W t, H t, O t ) denote the proportion of arrests for Black, White, Hispanic, and Other defendants at time t

18 Accounting for non-stationary distributions Notation Let t = 1, 2,..., T denote arrive times of cases to the NODA office Let A t be the race* of the screening ADA handling case arriving at time t Let D t be the race* of the defendant at time t Let p t = (B t, W t, H t, O t ) denote the proportion of arrests for Black, White, Hispanic, and Other defendants at time t Want to test: P{A t = i, D t = j} = P{A t = i}p{d t = j} for all t

19 The Algorithm Let M t denote a random variable taking values in the set {B, W, H, O} with probabilities p t Monte Carlo Random Assignment for t = 1,..., T : Set A t = Race of screening ADA at time t Sample D t M t Set X t = (A t, D t ) end for

20 The Algorithm Monte Carlo Random Assignment Each run of the algorithm generates a trajectory X = {X t } We generate many such trajectories X (1),..., X (B) The generated trajectories can be used to construct confidence intervals and calculate p-values

21 Results ADA Race DFDN Race Lower (2.5%) Upper (97.5%) Observed Value P-Value n W W W B W O W H B W B B B O B H O W O B O O O H Combined P-Value:.246 Table: Results from Monte Carlo simulations for screening assignment. The results are based on 5000 generated trajectories.

22 Results ADA Race DFDN Race Lower (2.5%) Upper (97.5%) Observed Value P-Value n W W W B W O W H B W B B B O B H O W O B O O O H Combined P-Value:.246 Table: Results from Monte Carlo simulations for screening assignment. The results are based on 5000 generated trajectories. Results are consistent with random assignment

23 Results Results are consistent with random assignment along Gender, overall and within type of crime at both screening and trial stages Race, overall and within type of crime at both screening and trial stages