Jun Zhang Department of Computer Science University of Kentucky

Transcription

1 Jun Zhang Department of Computer Science University of Kentucky

2 Background on Privacy Attacks General Data Perturbation Model SVD and Its Properties Data Privacy Attacks Experimental Results Summary 2

3 In applications of commerce, materials distribution, census, weather forecast, etc., data analysis becomes more and more popular and every important 3

4 Before we publish our data or give it to our collaborators, we have to protect the privacy of the data E.g., in social network, e.g., Facebook, private information such as someone s political opinion, interest, health, etc, could be leaked. (Insurance company rejected someone s s disability claims) E.g., if you apply for a job, and your would-be employer knows you have some health condition, such as HIV, and might reject you for that reason 4

5 Assume the original data is A, after perturbation, it becomes à =A+E, here E is perturbation noise We can keep A and E, and publish à The condition is that à must keep most patterns or information of A, A for future data mining analysis

6 Not necessarily! Note: Strictly speaking, if we can guess the values of A from those of Ã, à then we say that t à is not safe In numerical data privacy-preserving, preserving, it is usually difficult to figure out the exact values of A. However, using à and some background information, if we can restrict the values of A to a small interval, we also consider à as unsafe

7 Background on Privacy Attacks General Data Perturbation Model SVD and Its Properties Data Privacy Attacks Experimental Results Summary 7

8 For an n*m data matrix A, a general data perturbation mode can be represented as: Ã = AR + E, E Here, A is n*m original data matrix, Ã is the corresponding perturbed matrix, R is an orthogonal matrix, E is an n*m Gaussian noise matrix, i.e., E satisfies Gaussian distribution N(0,β 2 ),with 0 mean, and any standard deviation Many popular data perturbation model can be viewed as a special case of this General Data Perturbation Model 8

9 Data owner publishes the perturbed data matrix à for analysis, not the original matrix A, and keep matrices R and E in a safe place 9

10 If we know more about the noise matrix (e.g., its statistical distribution, mean, deviation, etc.), we are in a better position to figure out the original data For the general data perturbation model, just a little l bit information i can cause privacy breach 10

11 Ã is the perturbed dataset User ID 2005 Salary 2006 Salary 2007 Salary 2008 Salary , , , , ,854 37,874 30,847 35, ,859 79,125 80,241 78,840 Attacker:I know ID2409 is me. And I know my salary in 2005 was 71,821 (2005), 74,521 (2006), 78,239 (2007), and 79,235 (2008) 11

12 Given a know perturbed data Ã, If A i (the i-th row of the original matrix A, e.g. my true salary in each year) is known, we call A i Background Information Under this condition, we will show that: Other original data (e.g., A j, j i) in A can be breached/attacked 12

13 Given a know perturbed data Ã, If A i (the i-th row of the original matrix A, e.g. my true salary in each year) is known, we all A i Background Information Under this condition, we will show that: Other original data (e.g., A, j j i) in A can be breached/attacked 13

14 Background on Privacy Attacks General Data Perturbation Model SVD and Its Properties Data Privacy Attacks Experimental Results Summary 14

15 Eigenvalue, eigenvector Singular value decomposition (SVD) Stochastic analysis on matrix

16 A [n x m] = U [n x r] Λ [ r x r] (V [m x r] ) T A: n* m matrix (e.g.:n documents*m m keywords, n webpages*m links) U: n x r matrix (e.g.,:n documents, r topics) Λ: r x r diagonal matrix (strength of each topics) (r : rank of matrix A),some diagonal matrix is also denoted as Σ V: m x r matrix (e.g.,:m keywords,r topics) P1-16

17 Singular value decomposition (SVD) For an n*m data matrix A, its singular value decomposition is A = U ΛV T. Property: Under the Frobenius norm, the best rank-k approximation of A is A k = U k Λ k V T k min A-A k 17

18 A = U Λ V T - example:

19 Theorem [Press,92]: Any numerical matrix A can be factored as A = UΛV T, U, Λ, V: unique (*) () U, V: column orthogonal (i.e.,the columns of the matrices U and V have unit length, and they are mutually orthogonal) U T U = I; V T V = I (I: identify matrix) Λ: diagonal matrix,diagonal entries are nonnegative, and listed in a descending order

20 n A k = U k Λ κ V T k or, m Eckart-Young-Misky Theorem:Among all matrices ti of rank kk, A k is the closest to A in terms of the Frobenius norm = λ 1 u 1 v T u λ κ k λ 1 >= λ 2 >=... v T k

21 Av =λu A T u=λv A= U Λ V T AV = U Λ Av =λu A= U Λ V T A T = V Λ Τ U T A T = V Λ U T A T U= V Λ A T u=λv

22 A [n x m] = U [ n x r ] Λ [ r x r ] V T [ r x m] A [n x m] (A T ) [m x n] = U Λ 2 U T symmetric positive definite it (A T ) [ ] A [ ] =VΛ 2 V T symmetric (A ) [m x n] A [n x m] = V Λ V symmetric positive definite

23 ( (A T ) [m x n] A [n x m] ) k = V Λ 2k V T (A T A ) k ~ v 1 λ 1 2k v 1T when k>>1

24 (A T A ) k ~ v T 1 λ 2k 1 v 1T for k>>1 constant α,, (A T A ) k v ~ α v 1 This property can be used to numerically compute the eigenvectors

25 A [n x m] = U [ n x r ] Λ [ r x r ] V T [ r x m] A [n x m] v 1 [m x 1] = λ 1 u 1 [n x 1] u 1 T A = λ 1 v 1 T A T A v 2 1 = λ 12 v 1 The largest eigenvector is the fixed point of the symmetric matrix

26 A [n x m] = U [ n x r ] Λ [ r x r ] V T [ r x m] A [n x m] x [m x 1] = b [n x 1] x 0 = V Λ (-1) U T b Exact solution, approximate solution, or leastsquares solution

27 AR = RA = A A is the original matrix, R is an arbitrary orthogonal matrix Implication:Any i matrix multiplication li i with an orthogonal matrix (left or right multiplication) does not change its 2-norm y AR R A x

28 A= U Λ V T AV = U Λ Α i = U i Λ (Α i, the i-th column of the matrix Α) A= U Λ V T AV = U Λ A i V=U i Λ A i V = U i Λ Α i = UU i Λ Βecause V is an orthogonal matrix

29 A= U Λ V T AV = U Λ Α i = U i Λ (Α i, the i-th column of the matrix Α) A= U Λ V T AV = U Λ A i V=U i Λ A i V = U i Λ Α i = U i Λ A i V k = U i Λ k (V k, the first k columns of V) A i V=U i Λ A i V = U i Λ A i V k = U i Λ k

30 Backgrounds on Privacy Attacks A General Data Perturbation Model Review of Singular Value Decomposition and Its Properties Data Privacy Attacks Experimental Results Summary 30

31 Symbols Meaning A,E,à Original Oi i matrix, ti noise matrix, ti and modified perturbed matrix The i-th singular value of A, the i- th singular value of à σ i, ~ σ i A i A i,j1:j2 The i-th row of the matrix A In matrix A, the i-th row, from columns j 1 to j 2 A i k S The i-th row of the matrix A k The diagonal matrix in SVD of A

32 32

33 33

34 34

35 35

36 36

37 37

38 à A E = + y à i A i E i x The norm of E i cannot be greater than that of A i 38

39 y Ra i a i a i -Ra i Length is small x 39

40 40

41 y y à i E i à k i A i E i A k i E i k x x 41

42 42

43 y à k q A k q à k p A k p Meaning: The two angles are approximately equal x 43

44 y à k q A k q à k p A k p General attack model is Ã=AR+E. Here we used Ã=A+E. However, in geometric interpretation, R is only a rotation operation and does x not change the angle difference 44

45 45

46 Known Unknown 46

47 Known Unknown 47

48 Known Unknown This system has 4 equations, and 4 unknowns ( only the location of A 5 k, and the vector A 5 k has 4 dimensions), can be solved 48

49 Important question: How to know the conditions are satisfied? 49

50 In other words, under certain conditions, we have 50

51 Under certain conditions, According to [Boppana, 1987], the largest eigenvalue σ E of a Gauss noise matrix can be approximated by the square root of (n+m), when n>>m 51

52 How do we know if the conditions are satisfied? when σ ie is far smaller than 52

53 How do we know if the conditions are satisfied? when σ ie is far smaller than 53

54 If the following two conditions are satisfied The following system has a solution(exact or leastsquares solution to reduce the interval of uncertainty) Known Unknown 54

55 Background on Privacy Attack General Model for Data Perturbation SVD and Its Properties Data Privacy Attacks Experimental Results Summary 55

56 56

57 57

58 58

59 Background on Privacy Attacks General Model for Data Perturbation SVD and Its Properties Data Privacy Attacks Experimental Results Summary 59

60 What is privacy attack? Analyzing the effectiveness of privacy protection under the general model of data perturbation Why do we need to study privacy attacks? Numerical tables: Generality Privacy of numerical tables: Important (e.g., financial data) Background used for privacy attack: It can be easy to gain background information How to conduct privacy attack? Eigenvalue and eigenvector analysis Effective? Theoretical analysis and experimental results show that: For low and middle dimensional data (low and middle dimension: small m, large n, such that n/m ), the attack can be effective 60

61 61