Explaining Predictions from Data Argumentatively

Transcription

1 Explaining Predictions from Data Argumentatively Explain Workshop Ken Satoh 1 Oana Cocarascu Kristijonas Čyras Francesca Toni April 25, 2018 Department of Computing, Imperial College London, UK 1 National Institute of Informatics, Tokyo, Japan 1

2 Problem Examples/instances/cases DB = {e 1,...,e n } Example e = (F,o) DB consists of: (set of) features/attribute-value pairs/factors F = {f 1,...,f m } F label/class/outcome o L = {ϕ,ϕ} New example (N,?) features N F unknown label? Prediction: determine whether? = ϕ or? = ϕ Explain why 2

3 (Some) Existing Approaches To predict labels, could use Case-Based Reasoning (CBR) [Richter and Weber, 2013] Artificial Neural Networks (ANNs) [LeCun et al., 2015] etc. But may be hard to explain predictions [Andrews et al., 1995, Sørmo et al., 2005] hard to define formally showing similar examples need not suffice transparent/interpretable explanatory May also be data-hungry e.g. large DB needed 3

4 Our Approach Abstract Argumentation (AA) [Dung, 1995] deals with conflicting information AA-CBR [Čyras et al., 2016a]: AA-driven CBR models and deals with conflicting examples AA-CBR Explanations [Čyras et al., 2016b] debates explaining predictions ANNs with AA-CBR ANNs for feature selection AA-CBR predictions and explanations rule-based predictions and explanations 4

5 Feature Selection (ANN) Start with a training set E of examples (Y,o) features (of e.g. mushrooms 2 ) Y F E = {...,white,pink,red,crimson,maroon,...} label o L = {edible (ϕ), poisonous (ϕ)} Use autoencoder to get a trimmed dataset DB of examples {...,white,red,...} = F F E DB = {(Y,o) : (X,o) E, Y = X F} Ensure F leads to coherent DB (X,o X ),(Y,o Y ) DB, if X = Y, then o X = o Y DB is rational 2 archive.ics.uci.edu/ml/datasets/mushroom[dheeru and Karra Taniskidou, 2017] 5

6 Abstract Argumentation (AA) AA is used to create a model of DB. An AA framework is a graph (Args, ) Nodes: arguments Args represent information Edges: attacks represent conflicts Semantics determine good arguments E.g. grounded extension (set of arguments) a b c g d e f 6

7 AA-CBR Model From DB and ϕ construct (Args, ) with: Args = DB {({},ϕ)}; examples are arguments ({},ϕ) (being edible) is focus argument for (X,o X ),(Y,o Y ) DB {({},ϕ)}, it holds that (X,o X ) (Y,o Y ) iff 1. o X o Y, and (different outcomes) 2. Y X, and (specificity) 3. (Z,o X ) CB with Y Z X. (concision) 7

8 AA-CBR Model Graph (Mushrooms) ϕ is focus ({},ϕ) ({red,scaly},ϕ) ({red},ϕ) ({red, scaly, convex}, ϕ) 8

9 AA-CBR Prediction From DB, focus ϕ and (N,?) construct (Args N, N ) with: Args N = Args {(N,?)}; N = {((N,?),(Y,o Y )) : (Y,o Y ) Args and Y N}. (Args N, N ) extends (Args, ) with (N,?) attacking irrelevant examples Let G be the grounded extension of (Args N, N ). The AA-CBR prediction of (N,?) is: ϕ, if ({},ϕ) G; edible if focus argument is good ϕ, otherwise, if ({},ϕ) G. poisonous otherwise 9

10 AA-CBR Prediction Graph (Mushrooms) ϕ is focus (N,?) = ({red,convex},?) ({},ϕ) ({red,scaly},ϕ) ({red},ϕ) ({red, scaly, convex}, ϕ) G = {({red,convex},?),({red},ϕ)}. ({red,convex},?) ({},ϕ) G. So prediction is poisonous (ϕ). 10

11 AA-CBR Explanations Explanations of predictions are disputes between a proponent P (arguing for focus) and an opponent O (arguing against). Disputes as sub-graphs of (Args N, N ): Prediction is ϕ an explanation is any admissible dispute tree T for the focus argument ({},ϕ) every O node has a child no argument labels both P and O Prediction is ϕ an explanation is any maximal dispute tree T for the focus argument ({},ϕ) every O leaf is unattacked in (Args N, N ) 11

12 Explanation for Poisonous (Args N, N ) ϕ is focus ({},ϕ) T [P :({},ϕ)] ({red},ϕ) [O :({red},ϕ)] ({red,scaly},ϕ) [P :({red, scaly, convex}, ϕ)] ({red, scaly, convex}, ϕ) [O :({red,convex},?)] ({red,convex},?) 12

13 Explanation for Edible ϕ is focus ({},ϕ) [P :({},ϕ)] ({red},ϕ) [O :({red},ϕ)] ({red,scaly},ϕ) [P :({red, scaly, convex}, ϕ)] ({red, scaly, convex}, ϕ) ({red, scaly, convex, smooth},?) 13

14 Rules Logic programming rules from (Args, ) Alternative description of the model of DB Rule predictions coincide with AA-CBR predictions Alternative explanations of predictions Logic program P: For E : ({f 1,...,f m },o) Args, create a rule acc(e) f 1,...,f m,not acc(e 1 ),...,not acc(e k ). stating that E is accepted if all features f 1,...,f m apply, unless any of the attackers E 1,...,E k of E are accepted; Repeat for each attacker and its attackers in turn; For rule prediction, add features from N as facts to get P N. 14

15 Rules (Mushrooms) (Args, ) ϕ is focus ({},ϕ) E 1 : ({red},ϕ) E 2 : ({red,scaly},ϕ) E 3 : ({red,scaly,convex},ϕ) N = {red,convex} P: acc(focus) not acc(e 1 ). acc(e 1 ) red,not acc(e 3 ). acc(e 2 ) red,scaly,not acc(e 3 ). acc(e 3 ) red,scaly,convex. P N is P with red. convex. 15

16 Ongoing Work Datasets ANNs Categorical rather than binary features Multiple labels Rule simplification Related (argumentation-based) explanation concepts, e.g. [García et al., 2013, Fan and Toni, 2015, Schulz and Toni, 2016] Related (rule-based) explanation concepts, e.g. (neural) decision trees, inductive logic programming 16

17 Summary ML for feature selection within data Argumentation for model creation predictions rules dialectical and logical explanations 17

18 References i Andrews, R., Diederich, J., and Tickle, A. B. (1995). Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems, 8(6): Čyras, K., Satoh, K., and Toni, F. (2016a). Abstract Argumentation for Case-Based Reasoning. In Baral, C., Delgrande, J. P., and Wolter, F., editors, Principles of Knowledge Representation and Reasoning, 15th International Conference, pages , Cape Town. AAAI Press. Čyras, K., Satoh, K., and Toni, F. (2016b). Explanation for Case-Based Reasoning via Abstract Argumentation. In 6th International Conference on Computational Models of Argument, pages , Potsdam. IOS Press. Dheeru, D. and Karra Taniskidou, E. (2017). UCI Machine Learning Repository. Dung, P. M. (1995). On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-person Games. Artificial Intelligence, 77: Fan, X. and Toni, F. (2015). On Computing Explanations in Argumentation. In Bonet, B. and Koenig, S., editors, 29th AAAI Conference on Artificial Intelligence, pages , Austin, Texas. AAAI Press. 18

19 References ii García, A. J., Chesñevar, C., Rotstein, N., and Simari, G. R. (2013). Formalizing Dialectical Explanation Support for Argument-Based Reasoning in Knowledge-Based Systems. Expert Systems with Applications, 40: LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. Nature, 521(7553): Richter, M. and Weber, R. (2013). Case-Based Reasoning. Springer. Schulz, C. and Toni, F. (2016). Justifying Answer Sets Using Argumentation. Theory and Practice of Logic Programming, 16(1): Sørmo, F., Cassens, J., and Aamodt, A. (2005). Explanation in Case-Based Reasoning Perspectives and Goals. Artificial Intelligence Review, 24(2):