Characterization of Error Tradeoffs in Human Identity Comparisons: Determining a Complexity Threshold for DNA Mixture Interpretation

Transcription

1 Characterization of Error Tradeoffs in Human Identity Comparisons: Determining a Complexity Threshold for DNA Mixture Interpretation Boston University School of Medicine Program in Biomedical Forensic Sciences 72 E. Concord Street, Boston, MA Jacob S. Gordon Catherine M. Grgicak, Robin W. Cotton 21 February 2013 American Academy of Forensic Sciences 2013 Annual Meeting

2 FSF Emerging Forensic Scientist Award Paper Presentation

3 30 DNA Analyst: Is the suspect excluded as a mixture contributor? D8S1179 D21S11 D7S820 CSF1PO 16,17 29,32.2 8,13 11,12 AT = 30 RFU ST = 150 RFU D3S1358 TH01 D13S317 D16S539 D2S ,16 6,9 11,11 11,13 17, D19S433 13,15 vwa TPOX D18S51 15,17 8,13 13,17 30 AMEL D5S818 FGA 11,13 23,25 X 20

4 2012 International Symposium on Human Identification Mixture Interpretation Workshop Nashville, TN 15 October 2012

5 Statement of the Problem Interpretation Steps Step 2.5 DNA Commission of the International Society of Forensic Genetics: Recommendations on the Interpretation of Action Step 1 Identify the presence of a mixture Step 2 Designation of allelic peaks Step 3 Identify the number of contributors in the mixture Step 4 Estimation of the mixture proportion or ratio of the individuals contributing to the mixture Step 5 Consideration of all possible genotype combinations Step 6 Compare reference samples Possible to consider complexity threshold before proceeding? P. Gill, C.H. Brenner, J.S. Buckleton, A. Carracedo, M. Krawczak, W.R. Mayr, et al., DNA commission of the International Society of Forensic Genetics: recommendations on the interpretation of mixtures, Forensic Sci. Int. 160 (2006)

6 Generating Data for Error Analysis Simulated 10,000 mixtures Simulated subpopulation of 10,000 excluded individuals Simulated subpopulations of 10,000 included individuals for each mixture 10,000 individuals/mixture x 10,000 mixtures = 100,000,000 total included individuals Perturbed mixtures with increasing incidences of allelic drop-out 10,000 mixtures x 9 levels of drop-out = 90,000 additional mixtures 10,000 (No Drop-out) with Pr(D)=0.10 with Pr(D)=0.40 with Pr(D)=0.70 Simulated Database with Pr(D)=0.20 with Pr(D)=0.50 with Pr(D)=0.80 with Pr(D)=0.30 with Pr(D)=0.60 with Pr(D)=0.90 Simulated Excluded Suspects Database S 1 S 2 S 3 S n Simulated Included Suspects Database For mixture 1: S 1 For mixture 2: S 2 S 3 S n S 1 S 2 S 3 S n...

7 Relative Prevalence Modeling Considerations Peak height/area not modeled Allele detection is binary problem: Alleles either observed unequivocally or not at all Homozygous loci (for individuals) or overlapping, D16S539 contributed alleles (for mixtures) represented by relative prevalence (instead of peak height/area) Allelic dropout probability proportional to relative prevalence, Pr( D) L A Pr( D) Effects due to stutter ignored TPOX 8 Allele (# repeats) Simulated mixture contributor profiles summed in 1:1 ratio to produce simulated, two-person mixture profiles

8 Relative Prevalence Tabulating Allelic Discrepancies allelic discrepancy reference allele absent from mixture profile Reference Profile Mixture Profile Allelic Discrepancies 0 Reference Profile Mixture Profile 1 Reference Profile Mixture Profile 2 Allele (# repeats)

9 Types of Errors Mixture Truthfully Excluded Suspect Truthfully Included Suspect Analyst Action Reality Suspect is mixture contributor Suspect not mixture contributor Suspect not mixture contributor False Positive Type I Error True Negative Suspect is mixture contributor True Positive False Negative Type II Error

10 Cumulative Normalized Count Comparing Excluded Individuals to Simulated 10,000 excluded individuals compared to 10,000 mixtures 1x10 8 comparisons # Allelic Discrepancies

11 Cumulative Normalized Count Cumulative Normalized Count Proportion of Correct Exclusions When Pr(D) = 0: Correct exclusion ~100% of the time with fewer than 6 discrepancies # Allelic Discrepancies As Pr(D) increases: Correct exclusion rate increases Incorrect inclusion rate decreases Decreased likelihood of including a reference sample that ought to have been excluded as a potential contributor # Allelic Discrepancies

12 Cumulative Normalized Count Cumulative Normalized Count Proportion of Correct Inclusions When Pr(D) = 0: Correct inclusion 100% of the time # Allelic Discrepancies As Pr(D) increases: Correct inclusion rate decreases Incorrect exclusion rate increases Decreased likelihood of excluding a reference sample that ought to have been included as a potential contributor # Allelic Discrepancies

13 ROC Curve: Simulated 2-Person

14 ROC Curve: Simulated 2-Person Ideal Operating Point = (0,1) 0% False Positive Rate 0% False Negative Rate Increasing levels of drop-out result in higher rates of incorrect inclusions and exclusions

15 Complexity Threshold: Simulated

16 2012 International Symposium on Human Identification Mixture Interpretation Workshop Nashville, TN 15 October 2012

17 2012 International Symposium on Human Identification Mixture Interpretation Workshop Nashville, TN 15 October 2012

18 ROC Curve: Simulated 3-Person

19 Summary Analysis of mixtures and low-level samples carries pronounced risk of false inclusions and exclusions that increases with increasing levels of drop-out Decision whether mixture profile is suitable for interpretation can be made BEFORE comparison to reference profile Use ROC curves to: Determine complexity threshold / exclusion criteria Conditions under which interpretation of a mixture profile is not possible Establish laboratory s allelic discrepancy tolerance Number of allelic discrepancies that supports laboratory s specified error tolerances