Fundamental Limits of Database Alignment

Transcription

1 Fundaental Liits of Database Alignent Daniel Cullina Dept of Electrical Engineering Princeton University Prateek Mittal Dept of Electrical Engineering Princeton University Negar Kiyavash Dept of Electrical and Coputer Engineering Dept of Industrial and Enterprise Systes Engineering Coordinated Science Lab University of Illinois at Urbana-Chapaign Abstract We consider the proble of aligning a pair of databases with correlated entries We introduce a new easure of correlation in a joint distribution that we call cycle utual inforation This easure has operational significance: it deterines whether exact recovery of the correspondence between database entries is possible for any algorith Additionally, there is an efficient algorith for database alignent that achieves this inforation theoretic threshold F a 0,,,, 0,, 0 0,, 0, 0,,, 0 U a u u u 3 u n M U b v v v 3 v n F b, 0,, 0 0, 0,, 0 0,,, 0, 0, 0, 0 I THE DATABASE DEANONYMIZATION PROBLEM Suppose that we have two databases Each ite in the databases contain inforation about a single individual Soe individuals appear in both databases When a entry in the first database and an entry in the second database concern the sae individual, their contents are correlated The entries ay be two noisy observations of the sae signal, they ay be two copletely different types of data that have soe correlation through population statistics, or they ay even be correlated though the sapling process used to deterine which individuals appear in the database We consider the following question: If the databases are published with user identities reoved fro each entry, is it possible to learn the association between database entries that correspond to the sae individual by exploiting the correlation between the? Clearly, when there is enough correlation between entries about the sae individual and the databases are sall enough, it is possible to learn the true alignent between the database entries Our goal is to find the precise conditions under which it is possible to learn the coplete correspondence between entries with high probability In particular, we would like to deterine the easure of correlation that characterizes feasibility of perfect deanonyization in this setting This fraework for database alignent is related to several practical deanonyization attacks Narayanan and Shatikov linked an anonyized dataset of fil ratings to a publicly available dataset using correlations between the ratings [] Differential privacy has been widely used to quantifying privacy issues related to databases [] More recently, generative adversarial privacy has been proposed [3] In both cases, if users are present in ultiple databases, knowledge of alignent is required to fully apply these fraeworks Takbiri, Houansadr, Goeckel, and Pishro-Nik have recently investigated a closely related user privacy proble [4] Fig Two databases, F a and F b, with alphabets X a = X b = {0, } 4 and a atching M between their user identifier sets A Notation For finite sets X and Y, let R X Y be the set of real-valued atrices with rows indexed by X and coluns indexed by Y For x R X Y, let x k R X Y be the entry-wise power of x, ie the atrix such that x k i,j = x i,j k Let x k R X k Y k be the tensor power of x, ie the atrix such that for a X k and b Y k, x k a,b = k i=0 x a i,b i Let PX be the set of probability distributions on X B Foral description We have the following sets related to the user identifiers: U a U b M U a U b Set of user identifiers in the first database Set of user identifiers in the second database Bijective atching between the two types of user identifierss A bijection between U a and U b is a subset of U a U b in which each eleent of U a and U b appears exactly once The atching M contains the pairs of ids that correspond to the sae user The fact that M is a bijection iplies that M = U a = U b Throughout, we let n = M We have the following sets, functions, and distributions associated with the databases: X a X b F a : U a X a F b : U b X b F = F a, F b p PX a X b p a PX a p b PX b Figure illustrates a pair of databases Alphabet of entries in first database Alphabet of entries in second database First database Second database Joint distribution between related entries Marginal distribution on first alphabet Marginal distribution on second alphabet /8/$ IEEE 65

2 C Generative odel For each user u U a, there is a database entry F a u X a For a pair u, v M, the entries F a u and F b v are correlated via the joint distribution p: Pr[F a u = i, F b v = j M] = pi, j For distinct u, v U a, F a u and F a v are independent The sae is true for distinct u, v U b Thus we define rf a, f b ; = pf a u, f b v u,v so the joint distribution of the databases is Pr[F a = f a, F b = f b M = ] = rf a, f b ; D Relationship to graph alignent The ethods used in this paper are related to those used to analyze inforation theoretic thresholds for exact graph alignent [5] [7] An undirected graph G can be represented by its edge indicator function: V G {0, }, so we have a very siple type of inforation about each user pair The analogue to the generative odel is the correlated Erdős- Rényi distribution on graph pairs, where corresponding edge indicator rvs are sapled iid fro soe joint distribution on {0, } Once the arginal distributions are fixed, the one reaining degree of freedo specifies the level of correlation In the database proble, we instead have larger blocks of inforation about individual users This allows for ore coplicated fors of correlations In this paper, we identify the relevant one-diensional suary of that correlation A further connection is that graph alignent falls into the database alignent fraework when seed vertices are used [8], [9]: the list of adjacent seeds is essentially a database entry II RESULTS Both our achievability and converse bounds use the following easure of correlation in a joint distribution We propose to call this quantity cycle utual inforation Definition For p PX a X b, let z R Xa X b be the atrix such that z i,j = pi, j for i X a and j X b For an integer l, define the order-l cycle utual inforation I l p = l log trzzt l Then z has a singular value decoposition z = UΣV T where Σ = diagσ Observe that trσ = truσv T V ΣU T = trzz T = i,j z i,j =, so σ, the vector of squared singular values, constitutes a probability distribution Thus we have another expression for cycle utual inforation of order l: Il p = H lσ, where H l is the Rényi entropy of order l This expression allows us to extend the definition of Il p to all nonnegative real l Our achievability theore allows for arbitrary structure in the joint distribution of database entries Theore Let M U a U b be a uniforly rando bijection Let the alphabets X a and X a and the joint distribution p PX a X b depend on n If I p log n + ω, there is an estiator for M given F that is correct with probability o When the database entries are vectors of independent identically distributed coponents, we have a converse bound with a leading ter that atches the achievability Theore Let M U a U b be a uniforly rando bijection Fix alphabets Y a and Y b and a joint distribution q PY a Y b Let X a = Y l a, X b = Y l b, and p = q l, where l can depend on n If I p Ω log n, any estiator for M given F is correct with probability o III MAP ESTIMATION The optial estiator for M given F is the axiu a posteriori estiator: ˆf a, f b = argax Pr[M = F = f a, f b ] = argax a = argax Pr[F = f a, f b M = ] Pr[M = ] Pr[F = f a, f b ] Pr[F = f a, f b M = ] In a we use that fact that M is uniforly distributed Define the event E, = {f a, f b : rf a, f b ; rf a, f b ; } When is the true atching, this is the error event in which is incorrectly preferred to A Algorith for coputing the MAP estiator Define the atrix Qf a, f b R Xa X b, Qf a, f b u,v = log pf a u, f b v The MAP estiator is the ax weight atching in Qf a, f b : ˆf a, f b = argax Qf a, f b u,v u,v Thus ˆ can be coputed in On 3 tie [0] IV GENERATING FUNCTIONS Let x and y be two atrices of foral variables indexed by X a X b, and let x a and y a be vectors of foral variables indexed by X a, and let x b and y b be vectors of foral variables indexed by X b For a atching U a U b and a pair of databases f a : U a X a and f b : U b X b, define the generating function of the joint type t; f a, f b ; x = x fau,f b v u,v 65

3 Observe that t; f a, f b ; p = rf a, f b ; For a pair of atchings, define the generating function x, y = f a:u a X a f b :U b X b t ; f a, f b ; xt ; f a, f b ; y By understanding the behavior of this generating function, we can obtains upper bounds on the probability of an estiator aking an error Throughout this section, let z R Xa X b be a atrix and let z a R Xa and z b R X b be vectors such that z i,j = pi, j, z a i = p a i, and z b j = p b j Lea For any two bijections, U a U b, M = ] z, z Proof: For any θ 0, we have M = ] [ ] rfa, f b ; = E rf a, f b ; M = [ rfa ] θ, f b ; rf a, f b ; M = Furtherore, [ rfa ] θ, f b ; E rf a, f b ; M = = θ rfa, f b ; rf a, f b ; rf a, f b ; f a,f b rf a, f b ; θ rf a, f b ; θ t ; f a, f b ; p θ t ; f a, f b ; p θ t ; f a, f b ; p θ t ; f a, f b ; p θ = p θ, p θ where the atrix and vector exponents with are applied entrywise Selecting θ = gives the clai Define the generating function b l x, y = trxy T l Regard as a function X a X b and regard T as a function X b X a Then their coposition T is a perutation of X a Lea Let, U a U b be bijections Let t l be the nuber of cycles of length l in the perutation T Then t =, l lt l = X a, and x, y = l Nb l x, y t l Lea 3 For z R Xa X b with nonnegative entries and for l, b l z, z b z, z l/ Proof: We have b l z, z = k σl k where σ k are the singular values of z By a standard inequality on p-nors, k σl k l/ k k σ4 Lea 4 Let, U a U b be bijections and let d = n Then z, z b z, z d/ Proof: Fro, b z, z = Then the clai follows fro Leas and 3 V ACHIEVABILITY Proof of Theore : We will use a union bound over all possible errors [ ] Pr M = E, M = ] n = M = ] d= S,d where S,d is the set of atchings that differ fro is exactly d places We have n S,d d! n d d Fro Lea and Lea 4, we have M = ] l b l z, z t l b z, z l/ t l l = b z, z d/ Thus the overall probability of error is at ost n n d b z, z d/ d= Fro the ain condition of the theore, we have I p log n + ω b z, z exp log n ω = on, so for sufficiently large n, nb z, z / < and we have n n d b z, z d/ n b z, z nb o z, z/ d= which proves the clai 653

4 VI CONVERSE Lea 5 For any two bijections, U a U b, x, y = B, x, y Proof: For each l, b l x, y = b l y, x The perutations T and T are inverses and thus have the sae cycle decoposition The clai follows fro Lea Lea 6 Fix alphabets Y a and Y b and a joint distribution q PY a Y b Let l depend on n such that l = ω Let X a = Y l a, X b = Y l b, p = q l For any two bijections, U a U b such that = n, M = ] b z, z +o Proof: The function cθ = p θ, p θ is a conditional oent generating function: [ ] rfa, f b ; cθ = E exp θ log M = rf a, f b ; Fro Lea, we have because p θ, p θ = b p θ, p θ n b p θ, p θ = b p θ, p θ b p θ, p θ = trp θ p θ T = i,j By Lea 5 p θ i,jp θ i,j = cθ = b p θ, p θ = b p θ, p θ = c θ Moent generating functions are log-convex, so cθ is iniized at θ = Because p = q l, cθ is the product of l identical ters Let u = q θ and v = q θ b p θ, p θ = b u l, v l = tru l v l T u l v l T = truv T uv T l = b u, v l = b q θ, q θ l By Craér s Theore on the asyptotic tightness of the Chernoff bound [] [ ] rfa, f b ; Pr log 0 rf a, f b ; M = b q, q l o l = b p, p o Because l = ω, o l and o are equivalent Lea 7 For any three bijections,, 3 U a U b, E 3, M = ] b z, z d/ where d = n 3 Proof: For θ 0 and θ 0, [ rfa, f b ; Pr rf a, f b ; rf ] a, f b ; 3 rf a, f b ; M = = E[E 3, E, M = ] [ rfa θ θ ], f b ; rfa, f b ; 3 rf a, f b ; rf a, f b ; M = = θ θ rfa, f b ; rfa, f b ; 3 rf a, f b ; rf a, f b ; rf a, f b ; f a,f b rf a, f b ; θ rf a, f b ; 3 θ rf a, f b ; θ θ Choosing θ = θ =, we obtain E[E 3, E, M = ] f a,f b rf a, f b ; rfa, f b ; 3 = B, 3 z, z a b z, z d/ where a follows fro Lea 4 Proof of Theore : Let be the atching used to generate the databases and let S = S, be the set of atchings of size n that differ fro in exactly two places That is, for all S, = n Observe that S = n, because each eleent of S can be specified by the two users in U a that it atches differently than does Let X be the nuber of error events that occur: X = S E, Let ɛ = Pr[E, M = ], ie the probability that a specific transposition error occurs We need a lower bound on the probability that X > 0 Fro Chebyshev s inequality, we have [ Pr X E[X] E[X] ] [ ] X E[X] E[X] = E[X ] E[X] and we need to find conditions that ake this o We have X = E, E 3,, 3 S = E, + E, E 3, S {, 3} S For a set {, 3 } S, either 3 = n 3 or 3 = n 4 There are 3 n 3 pairs of the forer type and 3 n 4 pairs of the latter type In the latter case, the indicator variables E, and E 3, are independent In the forer case, let ɛ = E 3, M = ] Now we copute n n n n E[X] = ɛ = ɛ

5 and n n n E[X ] = ɛ + 6 ɛ + 6 ɛ 3 4 E[X ] E[X] n E[X] = ɛ ɛ + 6 n 3 ɛ ɛ n ɛ O n + ɛ ɛ nɛ Fro Lea 7 we have ɛ b z, z 3 and fro Lea 6 we have ɛ b z, z +o, so Pr[X = 0] O n b z, + z+o nb z, z +o If b z, z n +Ω, then n b z, z +o n ++o +Ω n Ω ω and Pr[X = 0] o VII PROPERTIES OF CYCLE MUTUAL INFORMATION Consider a joint distribution p PX a X b and recall the definitions of z and σ fro Section II The properties of σ reflect the correlation in the distribution p The following three conditions are equivalent: σ is supported on one point, the rank of the atrix z is one, and the p is the product of distributions on X a and X b Il p shares several properties with utual inforation It is syetric: Il p = I l pt It tensorizes: Il p k = kil p It reduces to entropy in the case of identical rando variables: if X a = X b and p = diagp, then I l diagp = H l p because σ is a rearrangeent of p In general, we have I l p inh l p a, H l p b Soething stronger is true: the distribution σ ajorizes p a and p b The diagonal of zz T is the arginal distributions p a : Furtherore, zz T i,i = j z i,j = j p i,j zz T i,i = UΣV T V ΣU T i,i = Ui,kσ k k Because U is an orthogonal atrix, the Hadaard product U U is doubly stochastic Thus σ ajorizes p a The diagonal of z T z contains p b, which is also ajorized by σ A Data processing inequality Lea 8 Let p PX, let q X PY, and let r Y PZ, so diagp PX X, diagpq PX Y, and diagpqr PX Z Then for integer l, I l diagpq I l diagpqr Proof: Define the atrices z i,k = diagpq i,k and w i,l = diagpqr i,l Then zz T i,i = ww T i,i = p i We have zz T i,j = p i p j qi,k q j,k k Y The su is the Bhattacharyya coefficient of the distributions q i, and q j,, which can be written in ters of the Bhattacharyya divergence as follows: exp D q i, q j, Siilarly ww T i,j = p i p j qr i,l qr j,l l Z By the data processing inequality for Rényi divergences [], we have Thus for all integer l D q i, q j, D qr i, qr j, zz T i,j ww T i,j trzz T l trww T l I l diagpq I l diagpqr ACKNOWLEDGEMENT This work was supported in part by NSF grants CCF 6-96, CCF 6-786, and CNS REFERENCES [] A Narayanan and V Shatikov, Robust de-anonyization of large sparse datasets, in IEEE Syposiu on Security and Privacy, 008, pp 5 [] C Dwork, Differential privacy: A survey of results, in International Conference on Theory and Applications of Models of Coputation Springer, 008, pp 9 [3] C Huang, P Kairouz, X Chen, L Sankar, and R Rajagopal, Contextaware generative adversarial privacy, Entropy, vol 9, no, p 656, 07 [4] N Takbiri, A Houansadr, D L Goeckel, and H Pishro-Nik, Matching Anonyized and Obfuscated Tie Series to Users Profiles, arxiv: [cs, ath], Sep 07 [5] D Cullina and N Kiyavash, Exact alignent recovery for correlated erdos-rényi graphs, 07 [6], Iproved achievability and converse bounds for Erdos-Rnyi graph atching, in Proceedings of the 06 ACM SIGMETRICS International Conference on Measureent and Modeling of Coputer Science ACM, 06, pp 63 7 [7] P Pedarsani and M Grossglauser, On the privacy of anonyized networks, in Proceedings of the 7th ACM SIGKDD international conference on Knowledge discovery and data ining ACM, 0, pp [8] S Ji, W Li, N Z Gong, P Mittal, and R A Beyah, On your social network de-anonyizablity: Quantification and large scale evaluation with seed knowledge in The Network and Distributed Syste Security Syposiu NDSS, 05 [9] O E Dai, D Cullina, N Kiyavash, and M Grossglauser, On the Perforance of a Canonical Labeling for Matching Correlated Erd\H{o}s-R\ enyi Graphs, arxiv: [cs, stat], Apr 08, arxiv: [0] J Edonds and R M Karp, Theoretical Iproveents in Algorithic Efficiency for Network Flow Probles, J ACM, vol 9, no, pp 48 64, Apr 97 [Online] Available: [] B Hajek, Rando processes for engineers Cabridge university press, 05 [] T Van Erven and P Harreos, Rényi divergence and Kullback-Leibler divergence, IEEE Transactions on Inforation Theory, vol 60, no 7, pp ,