Privacy in Statistical Databases

Transcription

1 Privacy in Statistical Databases Individuals x 1 x 2 x n Server/agency ) answers. A queries Users Government, researchers, businesses or) Malicious adversary What information can be released? Two conflicting goals Utility: Users can extract global statistics Privacy: Individual information stays hidden 1

2 Approach: Indistinguishability x 1 x 2. x n A local random coins queries answers ) x 1 x 2. x n A local random coins queries ) answers x is a neighbor of x if they differ in one row 2

3 Approach: Indistinguishability x 1 x 2. x n A local random coins queries answers ) x 1 x 2. x n A local random coins queries ) answers x is a neighbor of x if they differ in one row Neighboring databases induce close distributions on transcripts 2

4 Approach: Indistinguishability x 1 x 2. x n A local random coins queries answers ) x 1 x 2. x n A local random coins queries ) answers x is a neighbor of x if they differ in one row Definition: A is indistinguishable if, for all neighbors x, x, for all subsets S of transcripts Neighboring databases induce close distributions on transcripts Pr[Ax) S] 1 + ɛ)pr[ax ) S] 2

5 Approach: Indistinguishability ε has to be non-negligible here Distance measure on distributions matters This is a condition on the algorithm random process) A Saying this output is safe doesn t take into account how it was computed Definition: A is indistinguishable if, for all neighbors x, x, for all subsets S of transcripts Neighboring databases induce close distributions on transcripts Pr[Ax) S] 1 + ɛ)pr[ax ) S] 3

6 Differential Privacy Another interpretation [DM]: You learn the same things about me regardless of whether I am in the database Suppose you know I am the height of median Canadian You could learn my height from database! But it didn t matter whether or not my data was part of it. Has my privacy been compromised? No! Definition: A is indistinguishable if, for all neighbors x, x, for all subsets S of transcripts Neighboring databases induce close distributions on transcripts Pr[Ax) S] 1 + ɛ)pr[ax ) S] 4

7 Bayesian Interpretation For any given prior distribution, obtain almost) same posterior regardless of whether person i s data is included Proof : Bayes rule: posteriorx) = Indistinguishability implies red terms are almost the same regardless of whether person i s data is included. Definition: A is indistinguishable if, for all neighbors x, x, for all subsets S of transcripts PrAx) = transcript) priorx) PrAy) = transcript) priory) y Neighboring databases induce close distributions on transcripts Pr[Ax) S] 1 + ɛ)pr[ax ) S] 5

8 Other Properties Composition : If two institutions independently release information about me, using ε-indistinguishable algorithms A1 and A2 then the joint process A1 A2x) is 2ε-indistinguishable Group privacy ; ε-indistinguishability provides kεindistinguishability for the information about groups of size k Definition: A is indistinguishable if, for all neighbors x, x, for all subsets S of transcripts Neighboring databases induce close distributions on transcripts Pr[Ax) S] 1 + ɛ)pr[ax ) S] 6

9 Examples of Differential Privacy Output perturbation Sum queries [BDMN] Global Sensitivity [DMNS] Local sensitivity [NRS] Sample-Aggregate [NRS] Randomly break database into smaller datasets Compute something on each set Aggregate results for privacy Exponential Sampling [MT] General, abstract transformation on probability distributions More in development... Sofya Raskhodnikova may guest lecture on this Eric Shou s presentation 7

10 Example: Output Perturbation Individuals Server/agency x 1 x 2. x n A local random coins Tell me fx) fx) + noise User Two versions: Interactive: user asks for specific function f Non-interactive: server decides ahead of time on f1, f2, f3,... Intuition: fx) can be released accurately when f is insensitive to individual entries x 1, x 2,..., x n 8

11 Global Sensitivity Individuals x1 x 2. x n Server/agency A local random coins Tell me fx) fx) + noise User Intuition: fx) can be released accurately when f is insensitive to individual entries x 1, x 2,..., x n Global Sensitivity: Example: GS f = GS average = 1 n max neighbors x,x fx) fx ) 1 Theorem ) GSf If Ax) = fx) + Lap then A is ε-indistinguishable. ε 9

12 Examples of low global sensitivity Example: Sample mean average): Suppose x i [0, 1] Consider two neighboring datasets: How does fx) vary? meanx) = 1 n This holds for any neighboring datasets, so n i=1 GS mean = 1 n if x i [0, 1] x i fx) fx ) = x i n x i n 1 n x = x 1, x 2,..., x i 1, x i, x i+1,..., x n ) x = x 1, x 2,..., x i 1, x i, x i+1,..., x n ) 10

13 Examples of low global sensitivity Example: Sample mean average): GS mean = 1 n if x i [0, 1] Add noise Lap 1 ɛn ) meanx) = 1 n to meanx) in order to guarantee privacy Comparison: to estimate a frequency e.g. proportion of diabetics) in underlying population, get inherent error Why? There is a theoretical lower bound on the variance of any estimator for the expectation of an underlying distribution Noise added for privacy) << inherent error) T x) = meanx) + Lap 1 nɛ ) is an efficient estimator for the expectation n i=1 x i Θ 1 n ) 11

14 Global Sensitivity: Noise Distribution Theorem ) GSf If Ax) = fx) + Lap then A is ε-indistinguishable. ε Laplace distribution Lapλ) has density hy) e y 1 λ hy) y 12

15 Global Sensitivity: Noise Distribution Theorem ) GSf If Ax) = fx) + Lap then A is ε-indistinguishable. ε Laplace distribution Lapλ) has density hy) e y 1 λ hy+δ) hy) GSf Sliding property of Lap ε ) : hy) hy+δ) y eε δ GS f for all y, δ 13

16 Global Sensitivity: Noise Distribution Theorem ) GSf If Ax) = fx) + Lap then A is ε-indistinguishable. ε Laplace distribution Lapλ) has density hy) e y 1 λ hy+δ) hy) GSf Sliding property of Lap ε Proof idea: ) : hy) hy+δ) Ax): blue curve Ax ): red curve y eε δ GS f for all y, δ δ = fx) fx ) GS f 14

17 Examples of low global sensitivity Many natural functions have low GS, e.g.: Histograms and contingency tables On board last time) Covariance matrix on board) Distance to a property Functions that can be approximated from a random sample [BDMN] Many data-mining and learning algorithms access the data via a sequence of low-sensitivity questions e.g. perceptron, some EM algorithms, SQ learning algorithms This encompasses a huge variety of common algorithms My favorite: k-means clustering algorithm Problem: more queries more noise 15

18 Examples of Differential Privacy Output perturbation Sum queries [BDMN] Global Sensitivity [DMNS] Local sensitivity [NRS] Sample-Aggregate [NRS] Randomly break database into smaller datasets Compute something on each set Aggregate results for privacy Exponential Sampling [MT] General, abstract transformation on probability distributions More in development... Sofya Raskhodnikova may guest lecture on this Eric Shou s presentation 16

19 High Global Sensitivity: Median Example 1: median of x 1,..., x n [0, 1] x = 0 0 }{{} n }{{} n 1 2 x = 0 0 }{{} n }{{} n 1 2 medianx) = 0 medianx ) = 1 GS median = 1 Noise magnitude: 1. Too much noise! ε But for most neighbor databases x, x, medianx) medianx ) is small. Can we add less noise on good instances? 14 17

20 High Global Sensitivity: Cluster centers Database entries: points in a metric space. x x Global sensitivity of cluster centers is roughly the diameter of the space. But intuitively, if clustering is good, cluster centers should be insensitive. 18

21 High Global Sensitivity: Cluster centers Database entries: points in a metric space. x x Global sensitivity of cluster centers is roughly the diameter of the space. But intuitively, if clustering is good, cluster centers should be insensitive. 19

22 High Global Sensitivity: Cluster centers Database entries: points in a metric space. x x Global sensitivity of cluster centers is roughly the diameter of the space. But intuitively, if clustering is good, cluster centers should be insensitive. 20

23 Global versus local y y fy ) fy) fx ) x x f fx) adding noise Ax ) Ax) D n R d Distributions on R d Global sensitivity is worst case over inputs Local sensitivity: LS f x) = max x neighbor of x fx) fx ) 1 Reminder: GS f x) = max LS f x) x Goal: add less noise when local sensitivity is lower 21

24 Local Sensitivity LS f x) = Local sensitivity LS f x) = max fx) x neighbor of x fx ) 1 max x : neighbor of x fx) fx ) 1 Reminder: GS f = max LS f x) x Example: median for 0 x 1 x n 1, odd n x x m 1 x m x m+1 x 1 n median new median when x n = 0 new median when x 1 = 1 LS median x) = maxx m x m 1, x m+1 x m ) Goal: Release fx) with less noise when LS f x) is lower. 22

25 Instance-based noise: first attempt Can we have noise magnitude LS f x) instead of GS f? Problem: Noise magnitude might reveal information. Example: median x = 0 0 }{{} n } {{ 1} x = } 0 {{ 0} n 3 n medianx) = 0 medianx ) = }{{} n 3 2 LS median x) = 0 LS median x ) = 1 Pr[Ax) = 0] = 1 Pr[Ax) = 0] = 0 A is not ε-indistinguishable Lesson: Noise magnitude must be an insensitive function. 23

26 Smooth Bounds on Sensitivity Design sensitivity function Sx) Sx) is an ε-smooth upper bound on LS f x) if: for all x: Sx) LS f x) for all neighbors x, x : Sx) e ε Sx ) LS f x) x ) Sx) If Ax) = fx) + noise ε Theorem then A is ε -indistinguishable. Example: GS f is always a smooth bound on LS f x) 24

27 Smooth Bounds on Sensitivity Design sensitivity function Sx) Sx) is an ε-smooth upper bound on LS f x) if: for all x: Sx) LS f x) for all neighbors x, x : Sx) e ε Sx ) Sx) LS f x) x ) Sx) If Ax) = fx) + noise ε Theorem then A is ε -indistinguishable. Example: GS f is always a smooth bound on LS f x) 25

28 Smooth Bounds on Sensitivity Smooth sensitivity S f x)= max y LSf y)e ε distx,y)) Lemma For every ε-smooth bound S: S f x) Sx) for all x. Intuition: little noise when far from sensitive instances 26

29 Smooth Bounds on Sensitivity Smooth sensitivity S f x)= max y LSf y)e ε distx,y)) Lemma For every ε-smooth bound S: S f x) Sx) for all x. Intuition: little noise when far from sensitive instances y y x x D n 26

30 Smooth Bounds on Sensitivity Smooth sensitivity S f x)= max y LSf y)e ε distx,y)) Lemma For every ε-smooth bound S: S f x) Sx) for all x. Intuition: little noise when far from sensitive instances high local sensitivity y y low local sensitivity x x D n 26

31 Smooth Bounds on Sensitivity Smooth sensitivity S f x)= max y LSf y)e ε distx,y)) Lemma For every ε-smooth bound S: S f x) Sx) for all x. Intuition: little noise when far from sensitive instances high local sensitivity low smooth sensitivity y y low local sensitivity x x D n 26

32 Algorithmic Questions Applying this framework requires computing smooth bounds on sensitivity When can compute smooth bounds efficiently? How can we avoid this for complicated functions? What computational tasks can we solve privately? E.g.: learning and statistical inference are not exactly function evaluation 27

33 Computing Smooth Sensitivity Recall: Smooth sensitivity S f x) = max y Observation S fx) = max k=0,1,...,n e kε LS k fx) Example: median where LS k fx) = LSf y)e ε distx,y)) max LS f y). y:distx,y) k LS k median x) = max x m+t+k+1 x m+t ) t=0,1,...,k+1 x x m k 1 x m x m+k x n S median x) is computable in On2 ) time. 28

34 Computing Smooth Sensitivity Recall: Smooth sensitivity S f x) = max y Observation S fx) = max k=0,1,...,n e kε LS k fx) Example: median where LS k fx) = LSf y)e ε distx,y)) max LS f y). y:distx,y) k LS k median x) = max x m+t+k+1 x m+t ) t=0,1,...,k+1 x x m k 1 x m x m+k x n 29

35 Computing Smooth Sensitivity Recall: Smooth sensitivity S f x) = max y Observation S fx) = max k=0,1,...,n e kε LS k fx) Example: median where LS k fx) = LSf y)e ε distx,y)) max LS f y). y:distx,y) k LS k median x) = max x m+t+k+1 x m+t ) t=0,1,...,k+1 x x m k 1 x m x m+k x n 30

36 Computing Smooth Sensitivity Recall: Smooth sensitivity S f x) = max y Observation S fx) = max k=0,1,...,n e kε LS k fx) Example: median where LS k fx) = LSf y)e ε distx,y)) max LS f y). y:distx,y) k LS k median x) = max x m+t+k+1 x m+t ) t=0,1,...,k+1 x x m k 1 x m x m+k x n 31

37 Computing Smooth Sensitivity Recall: Smooth sensitivity S f x) = max y Observation S fx) = max k=0,1,...,n e kε LS k fx) Example: median where LS k fx) = LSf y)e ε distx,y)) max LS f y). y:distx,y) k LS k median x) = max x m+t+k+1 x m+t ) t=0,1,...,k+1 x x m k 1 x m x m+k x n 32

38 Computing Smooth Sensitivity Recall: Smooth sensitivity S f x) = max y Observation S fx) = max k=0,1,...,n e kε LS k fx) Example: median where LS k fx) = LSf y)e ε distx,y)) max LS f y). y:distx,y) k LS k median x) = max x m+t+k+1 x m+t ) t=0,1,...,k+1 x x m k 1 x m x m+k x n S median x) is computable in On2 ) time. 33

39 Computing Smooth Sensitivity If input is drawn i.i.d. from any distribution p on [0, Λ], then with high probability: S medianx) = O ) 1 nɛ + Λ Compare to: S meanx) Λ n e ɛn 2 So median has much better dependency on Λ, the diameter of the inputs domain Median is also more robust to outliers in data 34

40 Sample-and-Aggregate Methodology Intuition: Replace f with a less sensitive function f. fx) = gfsample 1 ), fsample 2 ),..., fsample s )) x x i1,..., x it x j1,..., x jt... x k1,..., x kt f6 f6 f6 6 aggregation function g noise calibrated to sensitivity of f + output 35

41 Sample-and-Aggregate Methodology Intuition: Replace f with a less sensitive function f. fx) = gfsample 1 ), fsample 2 ),..., fsample s )) x x i1,..., x it x j1,..., x jt... x k1,..., x kt Informally: Whatever information you can get f6 f6 f6 from a random subset of data points, 6 aggregation function g you can get with strong guarantees of privacy noise calibrated to sensitivity of f + output 36

42 Examples of Differential Privacy Output perturbation Sum queries [BDMN] Global Sensitivity [DMNS] Local sensitivity [NRS] Sample-Aggregate [NRS] Randomly break database into smaller datasets Compute something on each set Aggregate results for privacy Exponential Sampling [MT] General, abstract transformation on probability distributions More in development... Sofya Raskhodnikova may guest lecture on this Eric Shou s presentation 37

43 Lower Bounds Lower Bounds on privacy in general Dinur-Nissim: answering many questions accurately leads to a catastrophic breach of privacy many = On) accurately = with error on 1/2 ) catastrophic breach = adversary computes almost all values in the original database exactly We will see these bounds in more detail as an optional topic Lower bounds on differential privacy We can show stronger lower bounds on the power of noninteractive D.P. protocols future lecture) 38