Blind Attacks on Machine Learners Alex Beatson 1 Zhaoran Wang 1 Han Liu 1 1 Princeton University NIPS, 2016/ Presenter: Anant Kharkar NIPS, 2016/ Presenter: Anant Kharkar 1
Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 2
Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 3
Motivation Context: data injection attack (adversarial data added to existing distribution) Past work assumes attacker has knowledge of learner s algorithm (or can query for it) Here, consider both informed and blind attacker Statistical privacy - users may want to protect data via noise Objective: adversary makes it difficult to estimate distr. params NIPS, 2016/ Presenter: Anant Kharkar 4
Notation Distribution of interest: F θ density f θ, family F, data X i Malicious distribution: G φ density g φ, family G, data X i Combined dataset: Z, distribution P p(z) = αf θ (z) + (1 α)g φ (z) NIPS, 2016/ Presenter: Anant Kharkar 5
Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 6
Minimax Minimax risk - worst-case bound on population risk of estimator: M n = inf ˆψ sup E Z1:n Pψ n L(ψ, ˆψ n ) ψ Ψ Intuitively: minimum worst-case risk = minimum worst-case expected l2-norm KL-Divergence - deviation between two distributions Mutual information I (Z, V ) - measure of dependence between random variables NIPS, 2016/ Presenter: Anant Kharkar 7
Bounds Le Cam: Fano: [ 1 M n L(ψ 1, ψ 2 ) 2 1 ] nd 2 KL (P φ1, P φ2 2 [ M n δ 1 I (Z ] 1:n; V ) + log2 log V I (Z, V ) upper-bounded by D KL NIPS, 2016/ Presenter: Anant Kharkar 8
Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 9
Blind Attacker, Informed Learner Attacker knows F but not F θ, learner knows G φ Objective: maximize M n by choice of G φ φ = argmax φ M n = argmax φ inf ˆψ Minimize KL-Divergence ˆφ = argmin φ θ i V θ j V sup E Z1:n Pψ n L(ψ, ˆψ n ) ψ Ψ D KL (P θi,φ P θj,φ) V 2 n I (Z n ; θ) NIPS, 2016/ Presenter: Anant Kharkar 1
Blind Attacker, Blind Learner Learner does not know G φ, but knows G Ĝ = argmin G G = argmaxinf ˆθ (θ i,φ i ) V (θ j,φ j ) V sup (F θ,g φ ) F G E Z1:n L(θ, ˆθ) D KL (P θi,φ i P θj,φ j ) V 2 n I (Z n ; θ) NIPS, 2016/ Presenter: Anant Kharkar 1
Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 1
Informed Learner D KL (P i P j ) + D KL (P j P i ) α2 (1 α) F i F j 2 TV Vol(Z) Le Cam bound: ( ) 1 M n L(θ 1, θ 2 ) 2 1 2 α 2 2 (1 α) n F 1 F 2 2 TV Vol(Z) Fano bound: M n δ ( 1 α 2 (1 α) ) Vol(Z)nτδ + log2 log V Uniform attack bounds effective sample size at n α2 (1 α) Vol(Z) NIPS, 2016/ Presenter: Anant Kharkar 1
Outline 1 Introduction Motivation 2 Bounds Minimax Problem Scenarios 3 Results Informed Learner Blind Learner 4 Summary NIPS, 2016/ Presenter: Anant Kharkar 1
Informed Learner For α 1 2 - attacker can make learning impossible (KL-divergences sum to 0) Mimic attack: (G φ = F θ ) D KL (P i P j ) + D KL (P j P i ) (2α 1)2 (1 α) F i F j 2 TV 4 α4 1 α F 1 F 2 2 TV KL-divergence 0 as α 1 2 NIPS, 2016/ Presenter: Anant Kharkar 1
Summary Injection attacks against ML models 2 cases: blind learner, informed learner (attacker always blind) 2 attacks: uniform injection, mimic Attacker maximizes lower bounds on minimax risk NIPS, 2016/ Presenter: Anant Kharkar 1