Why on earth did you do that?!

Transcription

1 Why on earth did you do that?! A simple model of obfuscation-based choice Caspar Chorus Delft University of Technology Challenge the future

2 Background: models of decision-making Based on the notion that choices are based on motivations (preferences, desires, decision-rules) - Theory of Reasoned Action / Planned Behavior (Psych.) - Optimal Decision-Making (OR, Dec. Sciences) - Rational Choice Theory (Economics, Discrete Choice) - Belief-Desire-Intention framework (Multi-Agent Systems, AI) - In other words, motivations echo through in choices e.g. Revealed preference axiom Cornerstone of micro-econom(etr)ics 2

3 Research aim Current models of decision-making: Based on the notion that choices are based on motivations (preferences, desires, decision-rules) In other words, motivations echo through in choices This talk: decision-maker wishes to suppress the echo Exhibits choices, but aims to hide motivations from an onlooker Derivation of formal model of obfuscation-based decision-making + Illustration of selected properties Warning: work in (early) progress 3

4 Why obfuscate?! Human examples Protect privacy Hiding your preferences, motivations from other humans or from AI-powered systems (e.g. hide WtP for upgrade from KLM) Create moral wiggle room Faced with moral dilemma, unsure which moral principle to apply; choose so that it becomes difficult for others to judge you Create strategic ambiguity Avoid (legal, political, military) punishment by hiding motivations underlying actions; e.g. dual use nuclear technology 4

5 Why obfuscate?! AI examples Protect privacy Artificial agent / AI-system operating on behalf of human, e.g. in auction; seeks to preserve human s privacy Create moral wiggle room Autonomous, renegade AI with meta-intelligence wishes to avoid punishment from supervisor for, e.g. algorithmic discrimination Create strategic ambiguity Systems of AI-agents trying to hide strategic objections from each other in, e.g., military situations (autonomous weapons) 5

6 Relevance: Transportation Onlooker Obfuscator Human Artificial Human Moral decisions (e.g. taboo trade-offs) Keeping your Automated Vehicle on the moral high ground ( moral machine ) Artificial Recommender system (e.g. personal travel assistant) AV-AV interaction on multi-lane highways, crossroads 6

7 Relevance: Transportation (I) Onlooker Obfuscator Human Human Moral decisions (e.g. taboo trade-offs) Artificial Travel recommender Vehicle ownership tax Transport Policy 300 euro less tax system (e.g. personal travel Travel time assistant) 20 mins. less time Keeping your Automated Number of injured AV-AV interaction on 100 injured more Artificial Vehicle on the moral high ground ( moral machine ) Number of fatalities multi-lane highways, YOUR CHOICE crossroads 5 fatalities more I support I oppose 7

8 Relevance: Transportation (II) Onlooker Obfuscator Human Artificial Human Moral decisions by travelers and citizens (e.g. taboo trade-offs) Keeping your Automated Vehicle on the moral high Artificial Recommender system (e.g. personal travel assistant) AV-AV interaction on multi-lane highways, ground ( moral machine ) crossroads 8

9 Relevance: Transportation (III) Onlooker Obfuscator Human Artificial Human Moral decisions (e.g. taboo trade-offs) Keeping your Automated Vehicle on the moral high ground ( moral machine ) Artificial Recommender system (e.g. personal travel assistant) AV-AV interaction on multi-lane highways, crossroads 9

10 Relevance: Transportation Onlooker Obfuscator Human Artificial Human Moral decisions (e.g. taboo trade-offs) Keeping your Automated Vehicle on the moral high ground ( moral machine ) Artificial Recommender system (e.g. personal travel assistant) AV-AV interaction on multi-lane highways, crossroads 10

11 Relevance: variety of research fields Onlooker Obfuscator Human Artificial Human Artificial (moral) Psychology, Law, Artificial Intelligence, (geo-)politics, Expert Systems, (behavioural) Human-Computer econom(etr)ics Interaction Artificial Intelligence, Expert Systems, Multi-agent sytems Human-Computer Interaction 11

12 Intermezzo: obfuscation vs deception Why not assume that agents try to mislead? (rather than assuming that they try to obfuscate) Protect privacy No need to mislead (agent is not malicious) Create moral wiggle room Agent does not know the right moral rule Strategic ambiguity Deceit is more costly when found out 12

13 Base Model - single rule 13

14 Notation Set contains actions Set contains rules (alternatives, options) (motivations) by matrix contains scores describing how an action performs on a given rule. +,0, : obliged (+), permitted (0), prohibited ( ) Strong rule: +, Weak rule: 0, Agent chooses one action, follows one rule 14

15 ,, example e.g. obliged by, permitted by, prohibited by,. e.g. obliges, prohibits,. 15

16 Agent beliefs about onlooker 1. Exists, watches agent 2. Observes,, ; has same perception as agent 3. Has uninformative priors about agent s rule: =1 4. Observes agent s choice, uses it to update beliefs about rules using Bayes rule: Posterior probability that agent uses rule ", conditional on observing action #. = Probability that action # is chosen if rule " is followed! Prior probability 16

17 Probability that action # is chosen if rule " is followed Strong rule =1 if # is obliged under " =0 if # is prohibited under " Weak rule =1 % if # is permitted under ", where % is #actions that are permitted under ". =0 if # is prohibited under " 17

18 Agent behavior 1. Rule follower agent follows his rule 2. Full obfuscator agent does not care about rule, only about obfuscating the onlooker 3. Hybrid agent willing to give up rule-compliance if obfuscation-gain big enough 4. Costless obfusc. agent obfuscates within boundary of rule-compliance 18

19 Behavior of a Rule-follower Strong rule =1 if # is obliged under " =0 if # is prohibited under " Weak rule =1 % if # is permitted under ", where % is #actions that are permitted under ". =0 if # is prohibited under " 19

20 Behavior of a Full Obfuscator The agent knows that the supervisor s updated beliefs after having witnessed him action # result in updated beliefs How to quantify onlooker s uncertainty given updated beliefs: Using the notion of information Entropy (Shannon, 1948): & = ' log! Agent s behavior characterized by: argmax!.. & 20

21 Full obfuscator example e.g. obliged by, permitted by, prohibited by,. e.g. obliges, prohibits,. 21

22 Full obfuscator example (II) Action-probabilities conditional on following a particular rule P a r =1; P a r 1 =0.5; P a r 3 =0; P a r 4 = = = = =1 3 6 =6 =0 P a 1 r =0; P a 1 r 1 =0.5; P a 1 r 3 =0.5; P a 1 r 4 =1 6 =0; 6 =6 = 1 4 ; 6 = 1 2 P a 3 r =0; P a 3 r 1 =0; P a 3 r 3 =0.5; P a 3 r 4 =0 22

23 Full obfuscator example (III) Rule-posteriors conditional on choosing action 1, 2, 3 P a r =1; P a r 1 =0.5; P a r 3 =0; P a r 4 = = = = =1 3 6 =6 =0 P a 1 r =0; P a 1 r 1 =0.5; P a 1 r 3 =0.5; P a 1 r 4 =1 6 =0; 6 =6 = 1 4 ; 6 = 1 2 P a 3 r =0; P a 3 r 1 =0; P a 3 r 3 =0.5; P a 3 r 4 =0 6 =6 =6 =0;6 =1 23

24 Obfuscator worked out example (III) Entropy resulting from choosing action 1: & = 2 3 log log 1 = Entropy resulting from choosing action 2: & 1 = 1 4 log log log 1 2 Entropy resulting from choosing action 3: & 3 = 1 log 1 =0 =0.452 Action chosen by a full obfuscator 24

25 Full obfuscator example Isn t Full obfuscation the same as choosing the action that complies with the most rules? NO. 25

26 = A B = 0 > +? A + B # rules with which the action complies Entropy associated with choosing the action

27 Behavior of a Hybrid Agent cares about complying with his rule ("), but also wishes to prevent the onlooker from learning which rule that is Modeled e.g. using utility-maximization principles: C = 1 D + D & & FGH & FIJ & FGH If D=1 full obfuscator; if D=0 rule-follower NOTE: 0 < D willing to sacrifice rule-compliance 27

28 Costless obfuscation-behavior Agent wishes to prevent the onlooker from learning which rule governs his behavior, but not willing to sacrifice compliance argmax L M & L Where N denotes the set of alternatives permitted by the agent s rule 28

29 Costless obfuscation: Example Agent following 1 will prefer 1 over to obfuscate Agent following 3 will prefer 1 over 3 to obfuscate 29

30 Model extension: multi-rule 30

31 Notation Agent cares about multiple rules, to different degrees. Denote O the agent s utility associated with score (i.e., compliance or not of action # with rule ".) Simple specification, normalized: if # complies with ": O =P WeightP represents the relative importance of rule " to the agent (may be 0, negative). 31

32 Probability that action # is chosen given rule-weight vector Q One choice model could be Logit: -dimensional vector of P s Q = exp! O exp O!! This allows for an interpretation as a conventional choice model: Multi-attribute discrete choice experiment (DCE) Analyst wishes to infer tastes/weights for attributes By estimating a choice model based on observed choices Participant to DCE may wish to hide his tastes from analyst 32

33 Agent beliefs: multi-rule case 1. Exists, watches agent 2. Observes,, ; has same perception as agent 3. Has uninformative priors about agent s rule-weights: T Q 4. Observes agent s choice, uses it to update beliefs about weights using Bayes rule: Posterior beliefs about rule-weight vector, conditional on observing action #. T Q = Probability that action # is chosen given rule-weight vector Q T Q U Q T Q VQ W Prior beliefs 33

34 Rule-follower, Full obfuscator, Hybrid agent behavior Q = exp O X!!! exp O X Implicitly assumed in DCA Rule-follower max & = Y T Q log T Q W VQ Full obfuscator max C = 1 D Z [ \Z ]^_ Z ]`a \Z ]^_ +D b [ \b ]^_ b ]`a \b ]^_ Hybrid 34

35 Appendix: Active onlooker 35

36 Behavior of active supervisor Until now: Now: (implicit) assumption of passive supervisor: only exists in mind of the agent supervisor is able to determine the set (c) of actions from which the agent chooses. Select a set c of a given size d=e, which minimizes entropy ' & g c! k < ' & h g i h c h! c m Caveat: Entropy of action is contingent on set So, all choice-set compositions must be studied 36

37 Entropy of action is contingent on set example (single-rule) = = n = = n Same action, different choice set; different entropy 37

38 Single-rule, multi-rule onlooker o p [ q r s! & g! < & h gm k o p t qi r s h!! u! & g U g Q T Q VQ < & h g i U h g i Q T Q VQ v k h! u m v Note relation with experimental design for discrete choice analysis 38

39 Single rule case example w &, 1 =0.40 w & 1, 3 =0.38 y n, =B.B ' & g ' g!! = =0.4 39

40 (much) Work to be done 1. Theoretical: relax assumptions regarding mutual belief systems ( agent knows that onlooker knows that agent ) Game theory, Epistemic logic, Normative MAS 2. Empirical: to what extent and under what conditions does behaviour feature elements of obfuscation? Moral decisions, Negotiation support, Geopolitics 3. Econometrics: formulate within DCT, study identifiability of parameters (can D be identified, jointly with Ps?) 4. Simulations: norm formation in societies (inferring norms from actions) using Agent-based models 40