Database Privacy: k-anonymity and de-anonymization attacks

Transcription

1 18734: Foundations of Privacy Database Privacy: k-anonymity and de-anonymization attacks Piotr Mardziel or Anupam Datta CMU Fall 2018

2 Publicly Released Large Datasets } Useful for improving recommendation systems, collaborative research } Contain personal information } Mechanisms to protect privacy, e.g. anonymization by removing names } Yet, private information leaked by attacks on anonymization mechanisms 2

3 Non-Interactive Linking Background/A uxiliary Information DB1 DB2 Algorithm to link information De-identified record 3

4 Roadmap } Motivation } Privacy definitions } Netflix-IMDb attack } Theoretical analysis } Empirical verification of assumptions } Conclusion 4

5 Sanitization of Databases Add noise, delete names, etc. Real Database Sanitized Database Health records Census data Protect privacy Provide useful information (utility) 5

6 Database Privacy } Releasing sanitized databases 1. k-anonymity [Samarati 2001; Sweeney 2002] 2. Differential privacy [Dwork et al. 2006] (future lecture) 6

7 Re-identification by linking Linking two sets of data on shared attributes may uniquely identify some individuals: 87 % of US population uniquely identifiable by 5-digit ZIP, gender, DOB 7

8 K-anonymity } Quasi-identifier: Set of attributes that can be linked with external data to uniquely identify individuals } Make every record in the table indistinguishable from at least k-1 other records with respect to quasi-identifiers } Linking on quasi-identifiers yields at least k records for each possible value of the quasi-identifier 8

9 K-anonymity and beyond Provides some protection: linking on ZIP, age, nationality yields 4 records Limitations: lack of diversity in sensitive attributes, background knowledge, subsequent releases on the same data set l-diversity, m-invariance, t-closeness, 9

10 Re-identification Attacks in Practice Examples: } Netflix-IMDB } Movielens attack } Twitter-Flicker } Recommendation systems Amazon, Hunch,.. Goal of De-anonymization: To find information about a record in the released dataset 10

11 Roadmap } Motivation } Privacy definitions } Netflix-IMDb attack } Theoretical analysis } Empirical verification of assumptions } Conclusion 11

12 Anonymization Mechanism Delete name identifiers and add noise Gladiator Titanic Heidi Bob Alice Charlie Each row corresponds to an individual Each column corresponds to an attribute, e.g. movie Gladiator Titanic Heidi r r Anonymized Netflix DB 12 r

13 De-anonymization Attacks Still Possible } Isolation Attacks } Recover individual s record from anonymized database } E.g., find user s record in anonymized Netflix movie database } Information Amplification Attacks } Find more information about individual in anonymized database } E.g. find ratings for specific movie for user in Netflix database 13

14 Netflix-IMDb Empirical Attack [Narayanan et al 2008] Anonymized Netflix DB Gladiator Titanic Heidi r r r Publicly available IMDb ratings (noisy) Titanic Heidi Bob 2 1 Used as auxiliary information Weighted Scoring Algorithm Isolation Attack! r

15 Problem Statement Anonymized database Gladiator Titanic Heidi Auxiliary information about a record (noisy) r Titanic Heidi r Bob 2 1 r Attacker uses algorithm to find record Enhance theoretical understanding of why empirical de-anonymization attacks work 15 Attacker s goal: Find r 1 or record similar to Bob s record

16 Research Goal Characterize classes of auxiliary information and properties of database for which re-identification is possible 16

17 Roadmap } Motivation } Privacy definitions } Netflix-IMDb attack } Theoretical analysis } Empirical verification of assumptions } Conclusion 17

18 Netflix-IMDb Empirical Attack [Narayanan et al 2008] Anonymized Netflix DB Gladiator Titanic Heidi r r r Publicly available IMDb ratings (noisy) Titanic Heidi Bob 2 1 Used as auxiliary information How do you measure similarity of this record with Bob s record? (Similarity Metric) 18 Weighted Scoring Algorithm r What does auxiliary information about a record mean?

19 Definition: Asymmetric Similarity Metric Gladiator v 1 Titanic v 2 Heidi v 3 y r Intuition: Measures how closely two people s ratings match on one movie Movie (i) T(y(i), r(i)) Gladiator 0 Titanic 0.6 Individual Attribute Similarity y(i) r(i) T(y(i),r(i)) =1 p(i) 5 0 T( y(v 1 ), r(v 1)) = 1 = 0 5 p(i): range of attribute i Heidi 0 Similarity Metric Intuition: Measures how closely two people s ratings match overall 19 S(y,r) 0.6/2 = 0.3 S(y,r) = i supp(y) T(y(i),r(i)) supp(y) supp(y): non null attributes in y

20 Definition: Auxiliary Information Intuition: aux about y should be a subset of record y aux can be noisy e.g. Netflix r y sample aux captures information available outside normal data release process e.g. IMDb aux perturb Bound level of perturbation in aux γ [0,1] (m,γ)-perturbed auxiliary information i supp(aux).t(y(i),aux(i)) 1 γ supp(aux) = m = no. of non null attributes in aux 20

21 Weighted Scoring [Narayanan et al 2008, Frankowski et al 2006] Intuition: The fewer the number of people who watched a movie, the rarer it is Weight of an attribute i w(i) = 1 log( supp(i) ) supp(i) = no. of non null entries in column i Use weight as an indicator of rarity Score gives a weighted average of how closely two people match on every movie, giving higher weight to rare movies 21 Score( aux, rj) = Scoring Methodology supp(aux) = m = no. of non null attributes in aux Compute Score for every record r in anonymized DB to find out which one is closest to target record y w( i)* T( aux( i), rj( i)) i supp ( aux) supp( aux)

22 Weighted Scoring Algorithm [Narayanan et al 2008] Compute Score for every r in D w i v 1 v 2 v 3 r r r Score(aux, r j ) Score( aux, r j ) = w( i)* T( aux( i), r i supp ( aux) supp( aux) v 1 v aux One of the records r in anonymized database is y, which row is it? Eccentricity measure > threshold e(aux,d) = max r D(Score(aux,r)) max 2, r D(Score(aux,r)) j ( i)) 22 Output record with max Score r Score(aux, r) used to predict S(y,r)

23 Where do Theorems Fit? Computed: Score of all records r in D with aux Theorems help bridge the gap Desired: Guarantee about Similarity r r

24 Theorems } Theorem 1: When Isolation Attacks work? } Theorem 2: Why Information Amplification Attacks work? 24

25 Theorem 1: When Isolation Attacks work? Intuition: If eccentricity is high, algorithm always finds the record corresponding to auxiliary information! If aux is (m,γ)-perturbed Eccentricity threshold > γm Eccentricity: Highest score - Second highest score then Score(aux,r) = Score(aux,y) γ: Indicator of perturbation in aux M : Average of weights in aux r: Record output by algorithm y : Target record If r is the only record with the highest score then r = y 25

26 Isolation Attack: Theorem A. Datta, D. Sharma and A. Sinha. Provable De-anonymization of Large Datasets with Sparse Dimensions. In proceedings of ETAPS First Conference on Principles of Security and Trust (POST 2012) 26

27 Theorems } Theorem 1: When Isolation Attacks work? } Theorem 2: Why Information Amplification Attacks work? 27

28 Intuition: Why Information Amplification Attacks work? } If two records agree on rare attributes, then with high probability they agree on other attributes too } Use intuition to find record r similar to aux on many rare attributes (using aux as proxy for y) 28

29 Intuition: Why Information Amplification Attacks work? > 0.75 For > 90% of records } If a high fraction of attributes in aux are rare, then any record r that is similar to aux, is similar to y Similarity > 0.75 Similarity >

30 Theorem 2: Why Information Amplification Attacks work? Define Function f D ( η1, η2, η3) If a high fraction of attributes in aux are rare, then any record r similar to aux, is similar to y - Measure overall similarity between target record y and r that depends on: η : 1 η : 2 η3 : Fraction of rare attributes in aux Lower bound on similarity between r and aux Fraction of target records for which guarantee holds S( y, r) fd( η 1, η2, η3) 30

31 Theorem 2: Why Information Amplification Attacks work? Using Function f D ( η1, η2, η3) S( y, r) fd( η 1, η2, η3) Theorem gives guarantee about similarity of record output by algorithm with target record 31

32 Roadmap } Motivation } Privacy definitions } Netflix-IMDb attack } Theoretical analysis } Empirical verification of assumptions } Conclusion 32

33 Empirical verification } Use `anonymized' Netflix database with 480,189 users and 17,770 movies } Percentage values claimed in our results = percentage of records not filtered out because of } insufficient attributes required to form aux OR } insufficient rare or non-rare attributes required to form aux A. Datta, D. Sharma and A. Sinha. Provable De-anonymization of Large Datasets with Sparse Dimensions. In proceedings of ETAPS First Conference on Principles of Security and Trust (POST 2012) 33

34 Do Assumptions hold over Netflix Database? % Records for which Theorem 1 assumptions hold m = 10 m = 20 % Records Perturbation measure, gamma (γ) m : no. of attributes in aux Averaged over sample of records chosen with replacement 34 A. Datta, D. Sharma and A. Sinha. Provable De-anonymization of Large Datasets with Sparse Dimensions. In proceedings of ETAPS First Conference on Principles of Security and Trust (POST 2012)

35 Does Intuition about f D hold for Netflix Database? η : 1 η : 2 η : 3 f D ( η, η, 3) η Fraction of rare attributes in aux Lower bound on similarity between r and aux Fraction of target records for which guarantee holds can be evaluated given D f(n1,n2,0.9) n2:min value of S(aux,r) S( y, r) fd( η 1, η2, η3) Intuition verified n1:fraction of rare attributes in aux For Netflix DB, f D ( η 3) and 1, η2, η is monotonically increasing in η 1 η2 and tends to 1 as increases η 2 1

36 Roadmap } Motivation } Privacy definitions } Netflix-IMDb attack } Theoretical analysis } Empirical verification of assumptions } Conclusion 36

37 Conclusion } Naïve anonymization mechanisms do not work } We obtain provable bounds about, and verify empirically, why some de-anonymization attacks work in practice } Even perturbed auxiliary information can be used to launch de-anonymization attacks if: } Database has many rare dimensions and } Auxiliary information has information about these rare dimensions 37

38 Summary } Anonymity via sanitization } Offline sanitization } Online sanitization (next lecture) } Privacy definitions } k-anonymity } l-diversity }... 38

39 Summary } Deanonmyization attacks } Isolation } Amplification } Measuring attack success without ground truth } Assumptions } (m, γ)-perturbation } Measurables } similarity } eccentricity } η 1, η 2, η 3 39

40 Deanonymization Ground Truth Y y sanitization process modeling assumption Sanitized R Auxiliary Aux r aux 40

41 Isolation attack Ground Truth Y y isolation attack sanitization process modeling assumption (m, γ)-perturbation Sanitized R Auxiliary Aux r r aux 41

42 Amplification attack Ground Truth Y y* i 5 y 4 sanitization process amplification attack modeling assumption (m, γ)-perturbation Sanitized R Auxiliary Aux actual identity r* assumed identity r i 5 4 aux i??? 42 η 1 Fraction of rare attributes in aux η 2 Minimal similarity between r and aux η 3 Fraction of records satisfying of η 1, η 2

43 Offline/non-interactive release sanitized dataset k-anonymity Minimum anonymity set size Anonymization settings Privacy definitions Online/interactive sanitize queries l-diversity Minimum sensitive range size 43

44 Modeling Y -- Ground Truth records (NOT KNOWN) R -- Sanitized records = Obf(Y) Aux -- Auxiliary records (m, γ)-perturbation g.t./aux relationship Assumptions and Experimental Measurements Given aux in Aux, isolate r in R closest to it Deanonymization attacks Measurements e eccentricity best isolate r vs second best r η 1 Fraction of rare attributes in aux η 2 Minimal similarity between r and aux η 3 Fraction of records satisfying of η 1, η 2 Isolation Link auxiliary aux in A to r in R. Is aux is same identity as g.t. y à r? Amplification Use R to find values of fields not in aux Are predicted values close to g.t. y? Success estimation Theorem 1: Isolation success (m, γ), eccentricity à successful isolation Theorem 2: Amplification success η 1, η 2, η 3 à isolated r is close to g.t. y 44