Blind Attacks on Machine Learners

Transcription

1 Blind Attacks on Machine Learners Alex Beatson 1 Zhaoran Wang 1 Han Liu 1 1 Princeton University NIPS, 2016/ Presenter: Anant Kharkar NIPS, 2016/ Presenter: Anant Kharkar 1

2 Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 2

3 Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 3

4 Motivation Context: data injection attack (adversarial data added to existing distribution) Past work assumes attacker has knowledge of learner s algorithm (or can query for it) Here, consider both informed and blind attacker Statistical privacy - users may want to protect data via noise Objective: adversary makes it difficult to estimate distr. params NIPS, 2016/ Presenter: Anant Kharkar 4

5 Notation Distribution of interest: F θ density f θ, family F, data X i Malicious distribution: G φ density g φ, family G, data X i Combined dataset: Z, distribution P p(z) = αf θ (z) + (1 α)g φ (z) NIPS, 2016/ Presenter: Anant Kharkar 5

6 Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 6

7 Minimax Minimax risk - worst-case bound on population risk of estimator: M n = inf ˆψ sup E Z1:n Pψ n L(ψ, ˆψ n ) ψ Ψ Intuitively: minimum worst-case risk = minimum worst-case expected l2-norm KL-Divergence - deviation between two distributions Mutual information I (Z, V ) - measure of dependence between random variables NIPS, 2016/ Presenter: Anant Kharkar 7

8 Bounds Le Cam: Fano: [ 1 M n L(ψ 1, ψ 2 ) 2 1 ] nd 2 KL (P φ1, P φ2 2 [ M n δ 1 I (Z ] 1:n; V ) + log2 log V I (Z, V ) upper-bounded by D KL NIPS, 2016/ Presenter: Anant Kharkar 8

9 Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 9

10 Blind Attacker, Informed Learner Attacker knows F but not F θ, learner knows G φ Objective: maximize M n by choice of G φ φ = argmax φ M n = argmax φ inf ˆψ Minimize KL-Divergence ˆφ = argmin φ θ i V θ j V sup E Z1:n Pψ n L(ψ, ˆψ n ) ψ Ψ D KL (P θi,φ P θj,φ) V 2 n I (Z n ; θ) NIPS, 2016/ Presenter: Anant Kharkar 1

11 Blind Attacker, Blind Learner Learner does not know G φ, but knows G Ĝ = argmin G G = argmaxinf ˆθ (θ i,φ i ) V (θ j,φ j ) V sup (F θ,g φ ) F G E Z1:n L(θ, ˆθ) D KL (P θi,φ i P θj,φ j ) V 2 n I (Z n ; θ) NIPS, 2016/ Presenter: Anant Kharkar 1

12 Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 1

13 Informed Learner D KL (P i P j ) + D KL (P j P i ) α2 (1 α) F i F j 2 TV Vol(Z) Le Cam bound: ( ) 1 M n L(θ 1, θ 2 ) α 2 2 (1 α) n F 1 F 2 2 TV Vol(Z) Fano bound: M n δ ( 1 α 2 (1 α) ) Vol(Z)nτδ + log2 log V Uniform attack bounds effective sample size at n α2 (1 α) Vol(Z) NIPS, 2016/ Presenter: Anant Kharkar 1

14 Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 1

15 Informed Learner For α attacker can make learning impossible (KL-divergences sum to 0) Mimic attack: (G φ = F θ ) D KL (P i P j ) + D KL (P j P i ) (2α 1)2 (1 α) F i F j 2 TV 4 α4 1 α F 1 F 2 2 TV KL-divergence 0 as α 1 2 NIPS, 2016/ Presenter: Anant Kharkar 1

16 Summary Injection attacks against ML models 2 cases: blind learner, informed learner (attacker always blind) 2 attacks: uniform injection, mimic Attacker maximizes lower bounds on minimax risk NIPS, 2016/ Presenter: Anant Kharkar 1