DP-Finder: Finding Differential Privacy Violations by Sampling and Optimization. Dana Drachsler-Cohen

Transcription

1 DP-Finder: Finding Differential Privacy Violations by Sampling and Optimization Benjamin Bichsel Timon Gehr Dana Drachsler-Cohen Petar Tsankov Martin Vechev

2 Differential Privacy Basic Setting # disease 7 2

3 Differential Privacy Basic Setting # disease + noise 7.3 What about my privacy? 3

4 Differential Privacy - Intuition # disease + noise 7.3 or? Change my data # disease + noise 7.6 4

5 Differential Privacy More Abstractly x F F(x) Attacker check F(x) Φ? x Neighboring F F(x ) Attacker check F(x ) Φ? 5

6 Differential Privacy - Definition x ε-dp: F Pr[F x Φ] F(x) Pr[F(x ) Φ] Attacker check F(x) Φ? exp ε 1 + ε x Neighbouring Challenges induced by DP: Proving/checking ε-dp is hard F (buggy algorithms) F(x ) Attacker check F(x ) Φ? Proof strategies not complete Proofs only provide upper bounds 6

7 ε-dp Counterexamples (,, ) x x Φ that violate ε-dp: Pr[F x Φ] Pr[F(x ) Φ] > exp ε log Pr[F x Φ] Pr[F(x ) Φ] > ε 7

8 ε-dp Counterexamples (,, ) x x Φ that violate ε-dp: Pr[F x Φ] Pr[F(x ) Φ] > exp ε Maximize ε(x, x, Φ) log Pr[F x Φ] Pr[F(x ) Φ] > ε 8

9 Bounds on "true" ε Counterexample: 5%-DP Counterexample: 9.9%-DP Counterexample: 15%-DP Proven: 10%-DP (ε = 10% = 0.1) Evaluation: We get precise and large ε, close to known upper bounds 9

10 ε-dp Counterexamples Goal: Maximize ε(x, x, Φ) Challenge 1: Expensive to compute ε precisely Challenge 2: Search space is sparse: Few x, x, Φ lead to large ε(x, x, Φ) ε Estimate ε by sampling εƹ Make εƹ differentiable εƹ d 10

11 Step 1: Estimate ε ε εƹ Estimate ε by sampling 11

12 Estimating ε ε x, x Pr[F x Φ], Φ log Pr[F(x ) Φ] 12

13 Estimating ε ε x, x Pr[F x Φ], Φ log Pr[F(x ) Φ] n Pr F(x) Φ = 1 n i=1 i check F,Φ (x) x F(x) i check F,Φ (x) yes F 7.3 F 7.6 no 33% 67% F 6.8 yes 13

14 How precise is our estimate? Counterexample: 9.9% ± 10%-DP vs Counterexample: 9.9% ± DP Precision of ε Precision of Pr[F x Φ] and Pr[F x Φ] Sampling effort n Exponential search 14

15 Estimating precisely is expensive Probabillstic guarantees Heuristic Efficient Heuristic 10 4 Estimating ε up to an error of with confidence of 90% 15

16 Applying the M-CLT (Correlation) yes F 7.3 F 7.6 no 1 F 6.8 n i=1 n i check F,Φ x F F yes yes no 1 Follows 2D Gaussian distribution n n i=1 i check F,Φ x F 8.2 no 16

17 Obtaining a Confidence Interval for ε Joint likelihood of Pr[F x Φ] Pr[F x Φ] Likelihood of ε(x, x, Φ) Confidence Interval for ε(x, x, Φ) Distribution of Gauss Gauss (correlated): D. V. Hinkley On the Ratio of Two Correlated Normal Random Variables. Biometrika 56, 3 (1969),

18 How precise is our estimate? Counterexample: 9.9% ± 10%-DP vs Counterexample: 9.9% ± DP 18

19 Step 2: Finding Counterexamples εƹ εƹ d Make Ƹ ε differentiable 19

20 How can we optimize our estimate? maximize εƹ x, x, Φ = log 1 n σ i=1 n 1 n σ i=1 n i check F,Φ i check F,Φ (x) Not differentiable (x ) Goals Make differentiable Preserve semantics B 1 B B 1 B 2 B 1 B 2 if B x = E 1 else x = E 2 x = B E 1 + (1 B) E 2 20

21 How can we optimize our estimate? maximize εƹ x, x, Φ = log 1 n σ i=1 n 1 n σ i=1 n i check F,Φ i check F,Φ (x) Not differentiable (x ) Maximize using SLSQP (supports hard constraints for neighborhood) Random starting point (+ restart) What about division by zero? What about very small denominators? 21

22 Main differences to Ding et al. Dimension Ding et al. This work Problem statement ε x, x, Φ > ε 0? Maximize ε(x, x, Φ) Approach Statistical tests Estimate + confidence interval Search By patterns Gradient descent (incremental) 22

23 Evaluation How precise is the differentiable estimate? How efficient is DP-Finder in finding violations compared to random search? Exact solver (PSI) for ground truth 23

24 Precision of Differentiable Estimate ε Algorithms εƹ d ε 24

25 Random vs Optimized Random start Optimized 25

26 Ƹ Conclusion Differential Privacy ε-dp Counterexamples (,, ) Estimate ε ε εƹ Finding Counterexamples ε d εƹ 26