Small experiment. Netherlands Criminal Courts Prediction Machine. Netherlands Criminal Courts Prediction Machine

Transcription

1 Arguments for Structured Hypotheses: A Logico-Probabilistic Perspective Bart Verheij Artificial Intelligence, University of Groningen Legal tech exists, but is it disruptive? Disruption speak Richard Susskind: The Future of Lawyers: From Denial to Disruption IBM s Watson playing Jeopardy This 2-word phrase means the power to take private property for public use: it's ok as long as there is just compensation Harvard conference 2014 Disruptive Innovation in the Market for Legal Services Watson is almost certainly the most significant technology ever to come to law 1

2 Small experiment Watson is almost certainly the most significant technology ever to come to law Google: This 2-word phrase means the power to take private property for public use: it's ok as long as there is just compensation Netherlands Criminal Courts Prediction Machine Netherlands Criminal Courts Prediction Machine Predict Predict Let s push the button 2

3 Netherlands Criminal Courts Prediction Machine Netherlands Criminal Courts Prediction Machine Predict Let s push the button Predict Prediction: The suspect is guilty as charged Netherlands Criminal Courts Prediction Machine Predict Prediction: The suspect is guilty as charged This machine provides correct predictions in 95% of all cases (Cf. data collected by the Netherlands Bureau of Statistics) Questions and their answers Some questions have answers with a simple structure. Which 2-word phrase means the power to take private property for public use: it's ok as long as there is just compensation? Eminent domain What is Vincent van Gogh's country of birth? The Netherlands Is suspect John D guilty as charged? Yes Questions and their answers Other questions require answers with an elaborate structure. What are the requirements for our new office building? A requirements report How can I get to that wine bar in Shibuya? A plan, with a plan B Questions and their answers Simple answers: a word, phrase, sentence Complex answers: plans, explanations, arguments, interpretations, configurations Is the suspect guilty of the crime, and why? An explanatory, even justifying argument 3

4 A 1931 Wigmore chart Argumentation in Artificial Intelligence Umilian was accused of murdering Jedrusik. Toulmin s argument model Toulmin s argument model Harry was born in Bermuda So, presumably, Harry is a British subject D So, Q, C Since A man born in Bermuda will generally be a British subject Unless Both his parents were aliens/ he has become a naturalized American/... Since W On account of B Unless R On account of The following statutes and other legal provisions: D for Datum W for Warrant Q for Qualifier B for Backing Hitchcock, D., & B. Verheij (eds.) (2006). Arguing on the Toulmin Model. New C for Claim Essays in Argument Analysis and Evaluation. Argumentation R for Rebuttal Library, Vol. 10. Springer, Dordrecht. Hitchcock, D. & B. Verheij (2005). The Toulmin model today: Introduction to special issue of Argumentation on contemporary work using Stephen Edelston Toulmin's layout of arguments. Argumentation, Vol. 19, No. 3, pp Raymond Reiter proposes a formal model for default rules Pollock s red light example 1987, 1995 John Leslie Pollock proposes a computational model of defeasible argumentation 1995 Phan Minh Dung studies the mathematics of argument attack Undercutting defeat 4

5 Dung s abstract argumentation 1995 Dung s basic principle of argument acceptability The one who has the last word laughs best. Dung s basic principle of argument acceptability Dung s basic principle of argument acceptability The one who has the last word laughs best. The one who has the last word laughs best. Dung s basic principle of argument acceptability Dung s admissible sets α η ζ β δ γ ε The one who has the last word laughs best. Admissible, e.g.: {α, γ}, {α, γ, δ, ζ, η} Not admissible, e.g.: {α, β}, {γ} 5

6 Mary is owner John is owner Mary is original owner John is the buyer Verheij, B. (2005). Virtual Arguments. On the Design of Argument Assistants for Lawyers and Other Arguers. T.M.C. Asser Press, The Hague. Pros Cons Mary is owner John is owner Mary is owner John is owner Mary is original owner John is the buyer Mary is original owner John is the buyer John was not bona fide John was not bona fide Pros Cons Pros Cons John bought the bike for 20 State of the art in argumentation technology Argumentation semantics 2003 Today's argumentation technology is non-standard Cf. the history of the field Toulmin, Reiter, Pollock, Dung The connection of argumentation technology with standard techniques, in particular with logic and probability theory, has not been clarified. Verheij, B. (2003). DefLog: on the Logical Interpretation of Prima Facie Justified Assumptions. Journal of Logic and Computation 13 (3),

7 Open questions about argumentation The semantics question: How is argumentation connected to the world of facts and data? Today s argumentation models do not have a transparent connection to the world of facts and data Argument schemes (1) P. If P then Q. Therefore Q. (2) All Ps are Qs. Some R is not a Q. Therefore some R is not a P. The normative question: When are the process of argumentation and its outcomes acceptable? Today s argumentation models do not provide clear acceptability criteria Argument schemes (3) Person E says that P. Person E is an expert with respect to the fact that P. Therefore P. (4) Doing act A contributes to goal G. Person P has goal G. Therefore person P should do act A. Argument schemes Argument schemes are! context-dependent, not universal,! defeasible, not strict, and! concrete, not abstract. Are argument schemes hence a useless tool of analysis? No: take inspiration from knowledge engineering Critical questions Argument scheme for witness testimony: Witness A has testified that P. Therefore: P Argumentation, logic and probability theory Critical questions, for instance: Wasn t A mistaken? Wasn t A lying? 7

8 Argument strength as conditional probability Standard probability theory with its underlying classical logic Conclusions p(c 1 R) Reasons Other conclusions p(c 2 R) Kolmogorov: 1. p(φ) 0 for all φ in L. 2. If φ in L is a logical truth, then p(φ) = p(φ ψ) = p(φ) + p(ψ) for all φ in L and ψ in L such that φ and ψ are logically incompatible. Verheij, B. (2014). To Catch a Thief With and Without Numbers: Arguments, Scenarios and Probabilities in Evidential Reasoning. Law, Probability & Risk 13, Verheij, B. (2014). Arguments and Their Strength: Revisiting Pollock's Anti-Probabilistic Starting Points. Computational Models of Argument. Proceedings of COMMA 2014 (eds. Parsons, S., Oren, N., Reed, C., & Cerutti, F.), Amsterdam: IOS Press. Probability functions assume classical logic. Conditional probability p(ψ φ) := p(φ ψ) / p(φ) Argument strength reformulation of Bayes rule Strange property 8

9 Application: forensic science Designing and Understanding Forensic Bayesian Networks with Arguments and Scenarios The DNA effect By the success and nature of DNA the following idea gains momentum: Evidence is only valuable when it comes with scientifically supported statistics. (Cf. the CSI effect; conferences.law.stanford.edu/trialmath/ Three normative frameworks Goal: promote rational handling of evidence in courts Tool needed: a normative framework shared between experts and factfinders Probabilities E.g., follow the calculus, don t transpose conditional probabilities, don t forget prior probabilities Argumentation E.g., take all arguments into account, both pro and con, assess strength and relative strength, avoid fallacies Scenarios E.g., consider alternative scenarios, assess plausibility, consider which evidence is explained or contradicted 9

10 Three normative frameworks Probabilities: + Gradual uncertainty + Normative framework Bridge to legal context Argumentation: + Support and attack +/ Normative framework + Bridge to legal context Scenarios: + Global perspective ( holistic ) Normative framework + Bridge to legal context Argument Sjoerd Timmer (Utrecht) Floris Bex Probability Scenario Charlotte Vlek (Groningen) Three normative frameworks Argument Sjoerd Timmer (Utrecht) Floris Bex Probability Scenario Charlotte Vlek (Groningen) Probabilities: + Gradual uncertainty + Normative framework Bridge to legal context Argumentation: + Support and attack +/ Normative framework + Bridge to legal context Scenarios: + Global perspective ( holistic ) Normative framework + Bridge to legal context Argument strength as conditional probability Argument strength as conditional probability Conclusions Other conclusions p(c 1 R) p(c 2 R) One scenario Another scenario Reasons p(c 1 R) p(c 2 R) Verheij, B. (2014). To Catch a Thief With and Without Numbers: Arguments, Scenarios and Probabilities in Evidential Reasoning. Law, Probability & Risk 13, Verheij, B. (2014). Arguments and Their Strength: Revisiting Pollock's Anti-Probabilistic Starting Points. Computational Models of Argument. Proceedings of COMMA 2014 (eds. Parsons, S., Oren, N., Reed, C., & Cerutti, F.), Amsterdam: IOS Press. Evidence 10

11 A robbery A robbery A robbery H 1 H 2 H 3 H 4 H 5 H 6 H 7 H 8 H 1,..., H 8 : usual suspects E 1 E 1 : phone call E 1, E 2 T E 2 : surveillance camera E 3 : interrogation suspect 3 E 4 : interrogation suspect 7 E 5 : jewelry found E 1, E 2, E 3 E 1, E 2, E 3, E 4 E 1, E 2, E 3, E 4, E 5 J J T T T T: tattoo J: location of jewelry E 1 : phone call E 1 : phone call H i H i H j i j E 1 E 1 p(h i E 1 ) > 0, for each i. p(h i E 1 ) > 0, for each i. p(h i H j E 1 ) = 0, for each i and j with i j. 11

12 E 2 : surveillance camera E 2 : surveillance camera H i H j H i T E 1 E 2 E 1 E 2 p(h i E 1 E 2 ) > p(h j E 1 E 2 ), for i = 3 or 7, and each j 3 and 7. E 2 : surveillance camera From E 1 to E 5 H i T H 7 H 7 T J? 1 E 1 E 1... E 5 E 1 E 2 p(t H i E 1 E 2 ) = 1, for each i. Law as Argumentation One normative framework Probabilities AND Argumentation AND Scenarios Legal consequences (initial version) Pros Legal consequences (final version) + Gradual uncertainty + Normative framework Facts (initial version) Facts (final version) + Support and attack + Bridge to legal context Evidence (initial version) Cons Evidence (final version) + Global perspective ( holistic ) + Bridge to legal context 12

13 A Bayesian Network A Bayesian Network before court before court convicted convicted before court Not suspect before court convicted Not suspect convicted convicted 95% 0% before court 0.5% before court 100% ~0.025% convicted 0.475% Not suspect convicted 5% 100% Not suspect before court 99.5% Not suspect before court 0% ~99.975% Not suspect convicted % Designing and Understanding Forensic Bayesian Networks with Arguments and Scenarios How can hypothetical scenarios and the evidence for them be modeled in a Bayesian Network? Vlek, C., Prakken, H., Renooij, S., & Verheij, B. (2014). Building Bayesian Networks for Legal Evidence with Narratives: a Case Study Evaluation. Artificial Intelligence and Law 22 (4), Design method Given a collection of scenarios, we produce a Bayesian network modeling all scenarios. Legal idioms! Reusable modeling building blocks Fenton, Neil, Lagnado s legal idioms 1. Collect all relevant scenarios 2. Model each scenario using the scenario idiom 3. Merge these idioms with the merged scenarios idiom 4. Add evidence 13

14 Legal idioms! Narrative idioms Our project How can arguments be extracted from a Bayesian Network? Timmer, Sjoerd, Meyer, John-Jules, Prakken, Henry, Renooij, Silja & Verheij, Bart (2014). Extracting Legal Arguments from Forensic Bayesian Networks. Proceedings JURIX

15 Summary Conclusion Legal Information Technology Questions and answers Argumentation in Artificial Intelligence Argumentation, Logic and Probability Theory Application: Forensic Science Conclusion Argument strength as conditional probability Conclusions Other conclusions Arguments with structured hypotheses as conclusions p(c 1 R) p(c 2 R) One scenario Another scenario Reasons p(c 1 R) p(c 2 R) Verheij, B. (2014). To Catch a Thief With and Without Numbers: Arguments, Scenarios and Probabilities in Evidential Reasoning. Law, Probability & Risk 13, Verheij, B. (2014). Arguments and Their Strength: Revisiting Pollock's Anti-Probabilistic Starting Points. Computational Models of Argument. Proceedings of COMMA 2014 (eds. Parsons, S., Oren, N., Reed, C., & Cerutti, F.), Amsterdam: IOS Press. Evidence 15

16 Arguments for Structured Hypotheses: A Logico-Probabilistic Perspective Bart Verheij Artificial Intelligence, University of Groningen 16