The Optimal Mechanism in Differential Privacy

Transcription

1 The Optimal Mechanism in Differential Privacy Quan Geng Advisor: Prof. Pramod Viswanath 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 1

2 Outline Background on Differential Privacy ε-differential Privacy: Single Dimensional Setting ε-differential Privacy: Multiple Dimensional Setting (ε, δ)-differential Privacy Conclusion 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 2

3 Motivation Vast amounts of personal information are collected Data analysis produces a lot of useful applications 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 3

4 Motivation Big Data Release data/info. Applications Analyst How to release the data while protecting individual s privacy? 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 4

5 Motivation How to protect PRIVACY resilient to attacks with arbitrary side information? 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 5

6 Motivation How to protect PRIVACY resilient to attacks with arbitrary side information? Answer: randomized releasing mechanism 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 6

7 Motivation How to protect PRIVACY resilient to attacks with arbitrary side information? Answer: randomized releasing mechanism How much randomness is needed? completely random (no utility) deterministic (no privacy) 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 7

8 Motivation How to protect PRIVACY resilient to attacks with arbitrary side information? Answer: randomized releasing mechanism How much randomness is needed? completely random (no utility) deterministic (no privacy) Differential Privacy [Dwork et. al. 06]: one way to quantify the level of randomness and privacy 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 8

9 Background on Differential Privacy Collect data Users database Query Analyst Database Curator Released info. Database Curator: How to answer a query, providing useful data information to the analyst, while still protecting the privacy of each user. 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9

10 Background on Differential Privacy Collect data Users database Query Analyst Database Curator Released info. dataset: D query function: q (D) randomized released mechanism: K D Age A: 20 B: 35 C: 43 D: 30 q: How many people are older than 32? q(d) = 2 K(D) = 1 w.p. 1/8 K(D) = 2 w.p. 1/2 K(D) = 3 w.p. 1/4 K(D) = 4 w.p. 1/8 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 10

11 Background on Differential Privacy Two neighboring datasets: differ only at one element Age A: 20 B: 35 C: 43 D: 30 Age A: 20 B: 35 D: 30 D 1 D 2 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 11

12 Background on Differential Privacy Two neighboring datasets: differ only at one element Age A: 20 B: 35 C: 43 D: 30 Age A: 20 B: 35 D: 30 K D 1 K(D 2 ) D 1 D 2 Differential Privacy: presence or absence of any individual record in the dataset should not affect the released information significantly. 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 12

13 Background on Differential Privacy A randomized mechanism K gives ε-differential privacy, if for any two neighboring datasets D 1, D 2, and all S Range(K), Pr K D 1 S e ε Pr K D 2 S 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 13

14 Background on Differential Privacy A randomized mechanism K gives ε-differential privacy, if for any two neighboring datasets D 1, D 2, and all S Range(K), Pr K D 1 S e ε Pr K D 2 S Make hypothesis testing hard: e ε P FA + P MD 1 P FA + e ε P MD 1 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 14

15 Background on Differential Privacy A randomized mechanism K gives ε-differential privacy, if for any two neighboring datasets D 1, D 2, and all S Range(K), Pr K D 1 S e ε Pr K D 2 S ε quantifies the level of privacy ε 0, high privacy ε +, low privacy a social question to choose ε (can be 0.01, 0.1, 1, 10 ) 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 15

16 Background on Differential Privacy Q: How much perturbation needed to achieve ε-dp? A: depends on how different q D 1, q(d 2 ) are 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 16

17 Background on Differential Privacy Q: How much perturbation needed to achieve ε-dp? A: depends on how different q D 1, q(d 2 ) are If q D 1 = q D 2, no perturbation is needed. q D 1, q(d 2 ) 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 17

18 Background on Differential Privacy Q: How much perturbation needed to achieve ε-dp? A: depends on how different q D 1, q(d 2 ) are f D1 (x) f D2 (x) If q D 1 q D 2 is small, use small perturbation q D 1 q(d 2 ) 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 18

19 Background on Differential Privacy Q: How much perturbation needed to achieve ε-dp? A: depends on how different q D 1, q(d 2 ) are f D1 (x) If q D 1 q D 2 is large, use large perturbation f D2 (x) q D 1 q(d 2 ) 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 19

20 Background on Differential Privacy Global Sensitivity Δ: how different when q is applied to neighboring datasets Δ max q D 1 q D 2 (D 1,D 2 ) are neighbors Example: for a count query, Δ = 1 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 20

21 Background on Differential Privacy Laplace Mechanism: K D = q D + Lap( Δ ε ), Lap Δ ε is a r.v. with p.d.f f x = 1 2λ e x λ, λ = Δ ε f D1 (x) f D2 (x) Basic tool in DP 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 21

22 Optimality of Existing work? Laplace Mechanism: K D = q D + Lap( Δ ε ), Two Questions: Is data-independent perturbation optimal? Assume data-independent perturbation, is Laplacian distribution optimal? 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 22

23 Optimality of Existing work? Laplace Mechanism: K D = q D + Lap( Δ ε ), Two questions: Is data-independent perturbation optimal? Assume data-independent perturbation, is Laplacian distribution optimal? Our results: data-independent perturbation is optimal Laplacian distribution is not optimal: the optimal is staircase distribution 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 23

24 Part One: The Optimal Mechanism in ε-differential Privacy: Single Dimensional Setting presented in my preliminary exam 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 24

25 Recap of Part One: Problem Formulation minimize sup t R x R L x ν t(dx) s.t. ν t1 S e ϵ ν t2 S + t 1 t 2, measurable set S, t 1, t 2 R, s. t. t 1 t 2 Δ, L: R R ν t Δ cost function on the noise noise probability distribution given query output t global sensitivity 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 25

26 Recap of Part One: Main Results Optimality of query-output independent perturbation In the optimal mechanism, ν t is independent of t (under a technical condition). Staircase Mechanism Optimal noise probability distribution has staircase-shaped p.d.f. Minimize L x P dx s.t. Pr X S e ϵ Pr X S + d, d Δ, measurable set S 11/07/

27 Recap of Part One: Applications l 1 : L x = x l 2 : L x = x 2 V P γ = Δ eε 2 e ε 1 ε 0, the additive gap 0 ε +, V P γ = Θ(Δe ε 2) V Lap = Δ ε ε 0, the additive gap cδ 2 ε +, V P γ = Θ(Δ 2 e 2ε 3 ) V Lap = 2Δ2 ε 2 Huge improvement in low privacy regime 11/07/

28 Recap of Part One: Applications Properties of γ (for L x = x m ) Extension to Discrete Setting Extension to Abstract Setting 11/07/

29 Conclusion of Part One: Fundamental tradeoff between privacy and utility in ε-differential Privacy Staircase Mechanism, optimal mechanism for single real-valued query Huge improvement in low privacy regime Extension to discrete setting and abstract setting 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 29

30 Part Two: The Optimal Mechanism in ε-differential Privacy: Multiple Dimensional Setting 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 30

31 Multiple Dimensional Setting Query output can have multiple components q D = q 1 D, q 2 D,, q d D R d 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 31

32 Multiple Dimensional Setting Query output can have multiple components q D = q 1 D, q 2 D,, q d D R d If all components are uncorrelated, perturb each component independently to preserve DP. 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 32

33 Multiple Dimensional Setting Query output can have multiple components q D = q 1 D, q 2 D,, q d D R d If all components are uncorrelated, perturb each component independently to preserve DP. Composition Theorem: For each component q i (D) preserving ε-dp, end-to-end achieves dε-dp. 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 33

34 Multiple Dimensional Setting Query output can have multiple components q D = q 1 D, q 2 D,, q d D R d For some query, all components are coupled together. Histogram query: one user can affect only one component 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 34

35 Multiple Dimensional Setting Query output can have multiple components q D = q 1 D, q 2 D,, q d D R d For some query, all components are coupled together. Global sensitivity Δ max q D 1 q D 2 1 D 1,D 2 are neighbors Histogram query: Δ = 1 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 35

36 Problem Formulation: Query-output independent perturbation: K D = q(d) + X = (q 1 D + x 1, q 2 D + x 2,, q d D + x d ) 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 36

37 Problem Formulation: Query-output independent perturbation: K D = q(d) + X = (q 1 D + x 1, q 2 D + x 2,, q d D + x d ) Cost function on the noise: L: R d R 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 37

38 Problem Formulation: Query-output independent perturbation: K D = q(d) + X = (q 1 D + x 1, q 2 D + x 2,, q d D + x d ) Cost function on the noise: L: R d R Optimization problem: minimize L x 1,, x d P(dx 1 dx d ) R d s.t. P S e ε P S + t, S R d, t 1 Δ 38

39 Our Result: Optimality of Staircase Mechanism Theorem: If L x 1,, x d = x x d, d = 2, then optimal probability distribution has multidimensional staircase-shaped p.d.f. 39

40 Our Result: Optimality of Staircase Mechanism Theorem: If L x 1,, x d = x x d, d = 2, then optimal probability distribution has multidimensional staircase-shaped p.d.f. 40

41 Our Result: Optimality of Staircase Mechanism Theorem: If L x 1,, x d = x x d, d = 2, then optimal probability distribution has multidimensional staircase-shaped p.d.f. Conjecture: the result holds for any d 41

42 Proof Ideas Step 1: Argue only need to consider multi-dimensional piecewise constant p.d.f. by averaging Averaging preserves DP x 2 0 x 1 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 42

43 Proof Ideas Step 1: Argue only need to consider multi-dimensional piecewise constant p.d.f. by averaging Averaging preserves DP x 2 P1 e ε P5 P1 e ε P6 P1 e ε P16 P1 e ε P7 P1 e ε P15 P4 e ε P12 P4 e ε P13 P4 e ε P14 P4 e ε P15 P4 e ε P16 4 i=1 4 Pi e ε i=5 16 Pi 12 PhD Final Exam of Quan Geng, ECE, UIUC 43 x

44 Proof Ideas x 2 Averaging preserves DP Formulate a matrix filling-in problem row sum is a constant column sum is a constant 0 Explicit formula for d = 2, and arbitrary Δ 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 44

45 Proof Ideas Due to Step 1, the p.d.f. is a function of X 1 Step 2: Monotonicity Step 3: Geometrically Decaying Step 4: Staircase-Shape 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 45

46 Optimal γ for l 1 cost function (d=2) 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 46

47 Asymptotic analysis Laplacian Mechanism: V = 2Δ ε Multidimensional Staircase Mechanism: High privacy regime(ε 0): Low privacy regime(ε ): 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 47

48 Asymptotic analysis Laplacian Mechanism: V = 2Δ ε Multidimensional Staircase Mechanism: High privacy regime(ε 0): Low privacy regime(ε ): Independent Staircase Mechanism: V = Θ(2Δe ε 4) 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 48

49 Conclusion of Part Two: Extension of Optimality of Staircase Mechanism to Multi-dimensional Setting (d = 2) Conjecture: holds for any dimension Approximate Optimality of Laplacian Mechanism in High Privacy Regime Huge Improvement in Low Privacy Regime 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 49

50 Part Three: The Optimal Mechanism in (ε, δ)-differential Privacy 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 50

51 Part Three: (ε,δ)-differential privacy (ε, δ)-differential privacy Pr K D 1 S e ε Pr K D 2 S + δ Two special cases: (ε, 0)-differential privacy well studied in Part 1 and Part 2 (0, δ)-differential privacy 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 51

52 (ε,δ)-differential Privacy (ε, δ)-differential privacy Pr K D 1 S e ε Pr K D 2 S + δ Make hypothesis testing hard: e ε P FA + P MD 1 δ P FA + e ε P MD 1 δ 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 52

53 (ε,δ)-differential Privacy (ε, δ)-differential privacy Pr K D 1 S e ε Pr K D 2 S + δ Make hypothesis testing hard: e ε P FA + P MD 1 δ P FA + e ε P MD 1 δ Standard approach: adding Gaussian noise 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 53

54 (ε,δ)-differential Privacy (ε, δ)-differential privacy Pr K D 1 S e ε Pr K D 2 S + δ What is the optimal noise probability distribution in this setting? Our Results: (0, δ)-dp Optimality of Uniform Noise Mechanism (ε, δ)-dp Not much more general than (ε, 0)- and (0, δ)-dp Near-Optimality of Laplacian and Uniform Noise Mechanism in high privacy regime as ε, δ 0,0 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 54

55 Problem Formulation V min + i= L i P(i) s.t. P S e ϵ P S + d + δ, S Z, d Z, d Δ 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 55

56 (0,δ)-Differential Privacy V + min L i P(i) i= Theorem: If Δ = 1, assuming 1 2δ s.t. P S P S + d + δ, S Z, d Z, d Δ is an integer, optimal noise P.D. is P(i) δ 1 2δ 1 2δ 1 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC i 56

57 (0,δ)-Differential Privacy V + min L i P(i) i= Theorem: If Δ = 1, assuming 1 2δ 1 2δ s.t. P S P S + d + δ, S Z, d Z, d Δ is an integer, optimal noise P.D. is P(i) δ 1 2δ 1 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC i Proof idea: choose S = S k = {l: l k}, then P k δ, k 57

58 (0,δ)-DP: General Δ Upper Bound: Uniform Noise Mechanism V V UB 2 δ 2Δ 1 δ i=1 Δ L i + δ Δ L( Δ 2δ ) Δ 2δ V s.t. min + i= L i P(i) P S P S + d + δ, S Z, d Z, d Δ P(i) δ Δ Δ 2δ 1 i 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 58

59 (0,δ)-DP: General Δ Lower Bound: Duality of Linear Programming V s.t. min + i= L i P(i) P S P S + d + δ, S Z, d Z, d Δ choose S = S k = {l: l k}, then i=k k+δ 1 P(i) δ, k 59

60 (0,δ)-DP: General Δ Lower Bound: Duality of Linear Programming V s.t. min + i= L i P(i) P S P S + d + δ, S Z, d Z, d Δ choose S = S k = {l: l k}, then i=k k+δ 1 P(i) δ, k V LB = 2δ 1 2δ 1 L(1 + iδ) i=0 60

61 (0,δ)-DP: Comparison of V LB, V UB L(i) = i Constant additive gap 11/07/

62 (0,δ)-DP: Comparison of V LB, V UB L(i) = i Constant additive gap L(i) = i 2 Constant multiplicative gap 11/07/

63 (0,δ)-DP: Comparison of V LB, V UB L(i) = i Constant additive gap L(i) = L(i) = i 2 i m Constant multiplicative gap 11/07/

64 (ε,δ)-dp: Upper Bound V s.t. min + i= L i P(i) P S e ε P S + d + δ, S Z, d Z, d Δ Both (ε,0)-dp and (0,δ)-DP imply (ε,δ)-dp V min (V Lap, V Uniform ) Laplacian Mechanism: continuous: f x = 1 e x λ, λ = Δ 2λ ε discrete: P k = 1 e ε Δ 1+e ε Δ e k ε Δ 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 64

65 (ε,δ)-dp: Lower Bound V min + i= L i P(i) Lower Bound: Duality of Linear Programming choose S = S k = {l: l k}, by symmetry s.t. P S e ε P S + d + δ, S Z, d Z, d Δ 65

66 (ε,δ)-dp: Comparison of V LB, V UB L(i) = i, as ε, δ (0,0) min (V Lap, V Uniform ) 5.29 V LB V V UB = min (V Lap, V Uniform ) = Θ(min 1 ε, 1 δ ) 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 66

67 (ε,δ)-dp: Comparison of V LB, V UB L(i) = i, as ε, δ (0,0) min (V Lap, V Uniform ) V 5.29 LB V V UB = min (V Lap, V Uniform ) = Θ(min 1 ε, 1 δ ) L i = i 2, as ε, δ (0,0) min (V Lap, V Uniform ) 40 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC V LB V V UB = min (V Lap, V Uniform ) = Θ(min 1 ε 2, 1 δ 2 ) 67

68 (ε,δ)-dp: Multi-Dimensional Setting V min Z d L i 1, i 2,, i d P(i 1, i 2,, i d ) Upper Bound Uniform Noise Mechanism s.t. P S P S + d + δ, S Z d, d Z d, d 1 Δ Discrete Laplacian Mechanism P i 1,, i d = 1 λ 1 + λ 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC d λ i i d, λ = e ε/δ 68

69 (ε,δ)-dp: Multi-Dimensional Setting V min Z d L i 1, i 2,, i d P(i 1, i 2,, i d ) s.t. P S P S + d + δ, S Z d, d Z d, d 1 Δ Lower Bound 1. Choose certain sets S = S k m { i 1,, i d Z d i m k} 2. Write down Linear program, and derive the dual problem 3. Find proper dual variables to get lower bound 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 69

70 (ε,δ)-dp: Multi-Dimensional Setting Primal Problem Dual Problem 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 70

71 (ε,δ)-dp: Multi-Dimensional Setting (0,δ)-DP: l 1 l 2 (ε,δ)-dp: min (V Lap, V Uniform ) Uniform Noise Mechanism is Optimal when Δ = 1 C 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC V LB V V UB = min (V Lap, V Uniform ) C 9.49, for l 1 C 113, for l 2 71

72 Conclusion of Part Three Near-Optimality of Uniform Noise Mechanism in 0, δ -DP (ε, δ)-dp is not much more general than (0, δ)-dp and (ε,0)-dp min (V Lap, V Uniform ) C V LB V V UB = min (V Lap, V Uniform ) 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 72

73 Conclusion Fundamental tradeoff between privacy and utility in Differential Privacy Staircase Mechanism, optimal mechanism in ε-dp Huge improvement in low privacy regime Extension to Multi-Dimensional Setting Uniform Noise Mechanism, near-optimal mechanism in (0, δ)- DP (ε, δ)-dp is not much more general than (0, δ)-dp and (ε,0)-dp 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 73

74 Acknowledgment Thanks to my advisor, Prof. Pramod Viswanath 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 74

75 Thank you! 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 75