Adversarial Machine Learning: Big Data Meets Cyber Security

Transcription

1 Adversarial Machine Learning: Big Data Meets Cyber Security Bowei Xi Department of Statistics Purdue University With Murat Kantarcioglu

2 Introduction Many adversarial learning problems in practice. Image Classification/Object Recognition Intrusion Detection Fraud Detection Spam Detection Social Networks Adversary adapts to avoid being detected. New solutions are needed to address this challenge.

3 Million ways to write a spam From: "Ezra Martens" <ezrabngktbbem... To: "Eleftheria Marconi" <clifton@pu... Subject: shunless Phaxrrmaceutical Date: Fri, 30 Sep :49: Hello, Easy Fast = Best Home Total OrdeShipPrricDelivConf ringpingeseryidentiality VIAAmbCIALevVALXan GRAienLISitraIUMax $ $ $$ Get =additional informmation attempted to

4 Understanding Adversarial Learning It is not concept drift. It is not online learning. Adversary changes the objects under its control to avoid being detected. This is a game between the defender and the adversary.

5 Solution Ideas Constantly update a learning algorithm Dalvi et. al., Adversarial classification, KDD 2004 Optimize worse case performance Globerson and Roweis, Nightmare at test time: robust learning by feature selection, ICML 2006 Goals How to evaluate a defensive learning algorithm? How to construct a resilient defensive learning algorithm?

6 Game Theoretic Framework Often a classifier is updated after observing adversaries actions, such as spam filters. Adversarial Stackelberg Game Two players take sequential actions. 1. Adversary moves first by choosing a transformation T, f b (x) fb T (x) 2. After observing T, defender sets parameter values for a learning algorithm and creates a defensive rule h(x). f(x) = p g f g (x) + p b fb T (x). p g + p b = 1. Overall it is a mixture distribution with two classes.

7 Game Theoretic Framework Adversary s payoff is defined as u b (T, h) = L(h,g) g(t, x)f T b (x) dx. g(t, x) is profit for a bad object x being classified as a good one, after transformation T being applied. There is a penalty for transformation. L(h, g) is the region where the objects are identified as legitimate under decision rule h(x). Let C(T, h) be the misclassification cost. Defender s payoff is u g (T, h) = C(T, h). h T (x) is defender s best response facing T.

8 Game Theoretic Framework The adversary gain of applying transformation T is W (T ) = u b (T, h T ). W (T ) = L(h T,g) g(t, x)f T b (x)dx = E f T b (T e, h T e) is a subgame perfect equilibrium, s.t., T e = argmax T S ( W (T ) ). (I {L(hT,g)}(x)g(T, x)). After an equilibrium is reached, each player has no incentive to change their action.

9 Game Theoretic Framework Attribute selection is important in adversarial learning. Example: Gaussian mixture distribution, Naive Bayes classifier, and linear penalty for transformation. π g π b Penalty Equi. Bayes Error X 1 N(1, 1) N(3, 1) a = X 2 N(1, 1) N(3.5, 1) a = X 3 N(1, 1) N(4, 1) a =

10 Game Theoretic Framework Discretize the continuous numerical variables. Use the product of one dimensional marginal distributions to approximate the joint distribution. Search for an equilibrium using linear programming. Case study: Lending Club Data Peer to peer financing, offer small loans ranging from $1000 to $ Except for a credit report, many fields are easy to lie, such as job information, purpose of the loan etc.

11 Game Theoretic Framework We use the following attributes from the data set. X 1 Amount requested X 2 Loan purpose X 3 Debt-to-income ratio X 4 Home ownership (any, none, rent, own, mortgage) X 5 Monthly income X 6 FICO range X 7 Open credit lines X 8 Total credit lines X 9 Revolving credit balance X 10 Revolving line utilization X 11 Inquiries in the last 6 months X 12 Accounts now delinquent X 13 Delinquent amount X 14 Delinquencies in last 2 years X 15 Months since last delinquency

12 Game Theoretic Framework The instances originally are classified as: 1) Removed ; 2) Loan is being issued ; 3) Late ( days) ; 4) Late (16-30 days) ; 5) Issued ; 6) In review ; 7) In grace period ; 8) In funding ; 9) Fully paid ; 10) Expired ; 11) Default ; 12) Current ; 13) Charged Off ; 14) New instances after data cleaning. p g = p b = 0.5. Numerical attributes are discretized using 10 equi-width buckets. Naive Bayes classifier with independent attributes. Attacker transforms each attribute independently.

13 Game Theoretic Framework max subject to m i=1 j=1 ( m m f ij d ij + i=1 I {yi <q i } y i u i ) f ij 0 i = 1,..., m, j = 1,..., m (1) m f ij = p i i = 1,..., m (2) j=1 m f ij = y j j = 1,..., m (3) i=1 m y j = 1 (4) j=1 p i is the prob. of x = w i given that instance is in bad class. q i.is the prob. of x = w i given that instance is in bad class. f ij is the proportion moved from w i to w j by the adversary. d ij is the penalty of transforming X from w i to w j. u i is the expected profit. y i is the prob. after transformation.

14 Game Theoretic Framework The worst case scenario has transformation cost 0. For penalized case, we assume improving an attribute for one level costs $500 (results do not change in $300-$600 range). Attribute selection (choose top 6 attributes according to one dim classification accuracy after transformation) works well. Equilibrium performance is a good indicator of the long term success of a learning algorithm. Penalty prevents aggressive attacks. Experiment Type Accuracy Initial data set prior to transformation 89.95% Worst case scenario 61.25% Penalized transformation scenario 72.28% Attribute selection scenario 70.92%

15 Leader vs. Follower How defender being a leader versus being a follower affects its equilibrium strategies? One defender with utility D(h). m adversaries, each with utility A i (t i ). h and t i are player strategies.

16 Leader vs. Follower Defender being the leader A one-leader-m-follower game. 1. Given a leader s strategy h fixed, assume the m adversaries (i.e., the followers ) strategies are the attacks T = (t 1,, t m ). For the i-th adversary, further assume all other adversaries strategies are fixed, i.e., fixed t j, j i. Solve the following optimization for t h i : t h i = argmax {t i S i } {A i(t i, h)} 2. With the solution from above, T h = (t h 1,, th m) is the m adversaries joint optimal attacks for a given defender strategy h, the defender solves another optimization problem. h e = argmax {h H} {D(t h 1,, th m, h)} (h e, t he 1,, the m) is an equilibrium strategy for all players in the game.

17 Leader vs. Follower Defender being the follower A m-leader-one-follower game. 1. Given the joint attacks from the m adversaries, T = (t 1,, t m ), solve for the defender s optimal strategy. h T = argmax {h H} {D(t 1,, t m, h)} 2.With the solution above as the defender s optimal strategy h T against joint attacks T = (t 1,, t m ), solve for the optimal joint attacks T e. T e = (t e 1,..., te m) = argmax {ti S i, i} m i=1 A i (t i, h T ) (h T e, t e 1,, te m) is an equilibrium strategy for all players in the game.

18 Leader vs. Follower Gaussian Mixture Population The defender controls the normal population N p (µ g, Σ g ). Each adversary controls a population N p (µ b i, Σb i ). For an attack t, adversarial objects move toward µ g, the center of the normal. X t = µ g + (1 t) (X µ g ). 0 t 1. t = 1 is the strongest attack. Adversary population after attack is N((1 t)µ b + tµ g, (1 t) 2 Σ b ). m adversaries strategy is the joint attack (t 1,, t m ). Defender s strategy is to build a defensive wall around the center µ g.

19 Leader vs. Follower An Euclidean defensive wall is an ellipsoid shaped defensive region (X ˆµ g ) S g, 1 (X ˆµ g ) = χ 2 p (α). 0 < α < 1 controls the size. It is an approximate confidence region for multivariate normal. A Manhattan defensive wall forms a diamond shaped region. ˆσ g i p i=1 X i ˆµ g i ˆσ g i = η, is the sample standard deviation. ˆµ g i is the sample mean. η controls the size.

20 Leader vs. Follower Obtain η(α) as a function of α. Using the sample mean vector ˆµ g and var-cov matrix S g from the normal objects, generate a large sample from N(ˆµ g, S g ). For an η(α), (100α)% of the generated sample points fall into the diamond shaped Manhattan region with vertices at ˆµ g i ± η(α)ˆσg i on each dimension. A defender s strategy is to choose h = α or h = η(α). Defender utility is D(α) or D(η(α)). Let c be misclassification cost. D(α) = 100(error-of-normal + c error-of-adversary). L(t) = E ( max{k a X t X 2, 0} ), an adversary utility. k is max utility of an adversarial object. X t X 2 measures how much an adversary object is moved towards normal. Do exhaustive search to find the equilibria of the games.

21 Leader vs. Follower Euclidean defensive wall. Left is defender being the leader; right is defender being the follower.

22 Leader vs. Follower Manhattan defensive wall. Left is defender being the leader; right is defender being the follower.

23 Adversarial Clustering Previous adversarial learning work assumes the availability of large amounts of labeled instances. We develop a novel grid based adversarial clustering algorithm using mostly unlabeled objects with a few labeled instances. A classifier created using very few labeled objects is inaccurate. We identify 1) the centers of normal objects, 2) sub-clusters of attack objects, 3) the overlapping areas, 4) outliers and unknown clusters. We draw defensive walls around the centers of the normal objects to block out attack objects. The size of a defensive wall is based on the previous game theoretic study.

24 Adversarial Clustering It is not semi-supervised learning. We operate under very different assumptions. In adversarial settings, objects similar to each other belong to different classes, while objects in different clusters belong to the same class. Within a cluster, objects from two classes can overlap significantly. Adversaries can bridge the gap between two previously well separated clusters. Semi-supervised learning assigns labels to all the unlabeled objects aiming at the best accuracy. We do not label the overlapping regions and outliers.

25 Adversarial Clustering We have nearly purely normal objects inside the defensive walls, despite an increased Bayes error. Like in airport security. A small number of passengers use the fast pre-check lane, while all other passengers go through security check. The goal is not to let a single terrorist enter an airport, at a cost of blocking out many normal objects. A classification boundary is a defensive wall, analogous to a point estimate, while the overlapping areas analogous to confidence regions.

26 Adversarial Clustering A Grid Based Adversarial Clustering Algorithm Initialization: Set parameter values Merge 1: Creating labeled normal and abnormal sub-clusters Merge 2: Clustering the remaining unlabeled data points Merge 3: Merge all the data points without considering the labels Match: Match the big unlabeled clusters with the normal and abnormal clusters, and the clusters of the remaining unlabeled data points from the first pass. Draw α-level defensive walls inside the normal regions A weight parameter k controls the size of the overlapping areas.

27 Adversarial Clustering Compare with semi-supervised learning, EM least squares and S4VM. α = 0.6. True labels below. Blue is for normal; orange is for abnormal; purple for unlabeled; yellow and black for outliers

28 Adversarial Clustering Exp.1. Left is ADClust with k = 10; right is ADClust with k = 20.

29 Adversarial Clustering Exp.1. Left is EM least square; right is S4VM

30 Adversarial Clustering Exp.2. Left is ADClust with k = 10; right is ADClust with k = 20

31 Adversarial Clustering Exp.2. Left is EM least square; right is S4VM

32 Adversarial Clustering Experiment 1 has an attack taking place between two normal clusters. Experiment 2 has an unknown cluster, with normal and abnormal clusters heavily mixed under a strong attack. Our algorithm found the core areas of the normal. Semi-supervised wrongly labeled a whole normal cluster in experiment 1, and labeled the unknown cluster as normal in experiment 2. Smaller k is more conservative, creating larger unlabeled mixed areas.

33 Adversarial Clustering KDD Cup 1999 Network Intrusion Data Around 40 percent are network intrusion instances. We use instances from training data and top 7 continuous features. Experiment 1 has 100 runs. In one run, 150 instances are randomly sampled with their labels. 99.4% become unlabeled. Experiment 2 has 100 runs. In one run, 100 instances are randomly sampled with their labels. 99.6% become unlabeled. Larger k produces less unlabeled points, bigger normal regions containing more attack instances. More aggressive.

34 Adversarial Clustering Exp.1. Increase the weight k from 1 to 100. Top left is percent of abnormal points in mixed region;bottom left is percent of abnormal points among outliers; top right is the number of points in mixed region; and bottom right is the number of points as outliers.

35 Adversarial Clustering Exp.2. k equals to 1, 30 and 50 as low, medium and high weights. Set α levels from 0.6 to KDD data is highly mixed, yet we achieve on average nearly 90% pure normal rate inside the defensive walls

36 Adversarial SVM Classification boundaries. + is for the untransformed bad objects; o is for the good objects; * is for the transformed bad objects, i.e., the attack objects. The black dashed line is the standard SVM classification boundary, and the blue line is the Adversarial SVM classification boundary. Both the untransformed and the transformed bad objects are what we want to detect and block.

37 Adversarial SVM AD-SVM solves a convex optimization problem where the constraints are tied to adversarial attack models. Free-range attack: Adversary can move attack objects anywhere in the domain. C f (x min.j x ij ) δ ij C f (x max.j x ij ) The j th feature of an instance x i falls in between x max.j and x min.j. C f [0, 1]. C f = 0 means no attack. C f = 1 is the most aggressive attack.

38 Adversarial SVM Targeted attack: Adversary can only move attack instances closer to a targeted value. x t i is the target. The adversary adds δ ij to the j th feature x ij of an attack object. δ ij x t ij x ij. An upper bound on the amount of movement for x ij : 0 (x t ij x ij)δ ij C ξ x t ij 1 C x ij δ x ij + x t ij (x t ij x ij) 2 C δ reflects the loss of malicious utility. It can be larger than 1. 1 C δ x t ij x ij x ij + x t ij is the maximum percentage of x t ij x ij that δ ij is allowed to be. C ξ [0, 1] is a discount factor. Larger C ξ allows more movement.

39 Adversarial SVM SVM risk minimization model: free-range attack argmin 1 w,b,ξi,t i,u i,v i 2 w 2 + C i ξ i s.t. ξ i 0 ξ i 1 y i (w x i + b) + t i t i ( j C f vij (x max j u i v i = 1 2 (1 + y i)w u i 0 v i 0 x ij ) u ij (x min j x ij ) ) SVM risk minimization model: targeted attack argmin 1 w,b,ξi,t i,u i,v i 2 w 2 + C i ξ i s.t. ξ i 0 ξ i 1 y i (w x i + b) + t i t i j e ij u ij ( u i + v i ) (x t i x i) = 1 2 (1 + y i)w u i 0 v i 0

40 Adversarial SVM spambase data is from the UCI data repository. Accuracy of AD- SVM, SVM, and one-class SVM on the spambase data as free range attacks intensify. C f increases as attacks become more aggressive. Generate attacks as δ ij = f attack (x t ij x ij) f attack = 0 f attack = 0.3 f attack = 0.5 f attack = 0.7 f attack = 1.0 AD-SVM C f = AD-SVM C f = AD-SVM C f = AD-SVM C f = AD-SVM C f = SVM One-Class SVM

41 Adversarial SVM C δ decreases as targeted attacks become more aggressive. C ξ = 1 f attack = 0 f attack = 0.3 f attack = 0.5 f attack = 0.7 f attack = 1.0 AD-SVM C δ = AD-SVM C δ = AD-SVM C δ = AD-SVM C δ = AD-SVM C δ = SVM One-class SVM AD-SVM is more resilient to modest attacks than other SVM learning algorithms.

42 Funding Support ARO W911NF : Data Analytics for Cyber Security: Defeating the Active Adversaries, UT Dallas PI: M. Kantarcioglu, Purdue PI: B. Xi, $470,000, 08/07/ /06/2020, Amount Responsible: $222,500 ARO W911NF : A Game Theoretic Framework for Adversarial Classification, UT Dallas PI: M. Kantarcioglu, Purdue PI: B. Xi, $440,000, 08/01/ /31/2015, Amount Responsible: $210,140 Publications Kantarcioglu, M., Xi, B., and Clifton, C., Classifier Evaluation and Attribute Selection against Active Adversaries, Data Mining and Knowledge Discovery. 22(1-2), , 2011 Zhou, Y., Kantarcioglu, M., Thuraisingham, B., and Xi, B., Adversarial Support Vector Machine Learning, Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, , KDD 2012

43 Kantarcioglu, M., and Xi, B., Adversarial Data Mining, North Atlantic Treaty Organization (NATO) SAS-106 Symposium on Analysis Support to Decision Making in Cyber Defense, 1 11, June, 2014, Estonia Zhou, Y., Kantarcioglu, M., Xi, B., (Hot Topic Essay) Adversarial Learning: Mind Games with the Opponent, ACM Computing Reviews, August 2017 Zhou, Y., Kantarcioglu, M., Xi, B., A Survey of Game Theoretic Approach for Adversarial Machine Learning, revision submitted Wei, W., Xi, B., Kantarcioglu, M., Adversarial Clustering: A Grid Based Clustering Algorithm against Active Adversaries, submitted Zhou, Y., Kantarcioglu, M., Xi, B., Adversarial Active Learning, submitted