Prompt Network Anomaly Detection using SSA-Based Change-Point Detection. Hao Chen 3/7/2014

Transcription

1 Prompt Network Anomaly Detection using SSA-Based Change-Point Detection Hao Chen 3/7/2014

2 Network Anomaly Detection Network Intrusion Detection (NID) Signature-based detection Detect known attacks Use pattern for detection Very high accuracy Anomaly detection Detect unknown attacks, or zero-day attacks The essential rational of detection: Detecting any network behavior which is distinguishable from normal activities. Tolerate certain false positive

3 Attack Example: Witty Worms 2004 More than Five Days Monitoring Variation of Destination IPs VS. specified Source IP flows

4 Attack Example: Flooding DDoS 2007 Around One Hour s Monitoring Variation of Source IPs VS. Victim s IP

5 Attack Example: Shrew DDoS attack It uses short synchronized bursts of traffic to disrupt TCP connections on the same link, by exploiting a weakness in TCP's re-stransmission timeout mechanism

6 Change-Point Detection Change-point detection concerns with the design and analysis of procedures for on-thego detection of possible changes in the characteristics of a running (random) process. Change-point The time instance at which the state of the process changes Purpose : to detect changes in the statistical profile of network traffic as rapidly as possible, while maintaining a tolerable level of the risk of making a false detection.

7 The Relation of SSA-based Change-Point Detection Statistical Inference CUSUM Maximum Likelihood approach Shiryaev-Roberts (SR) Procedure EWMA Bayesian approach Depends on the change-point s prior distribution. However, it is unknown Singular Spectral Analysis (SSA) In time series analysis, singular spectrum analysis (SSA) is a nonparametric spectral estimation method. *. When the likelihood ratio is either difficult or impossible to obtain or compute, it can be replaced with a score function, S n (X 1,..., X n ). SSA-based Score Function SSA relates to the spectrum of eigenvalues in a Singular Value Decomposition(SVD) of a covariance matrix

8 SSA-based Score Function The main idea of SSA M-dimensional observations (columns of the trajectory matrix) are projected onto a subspace of Eigenvectors, which provides a way to learn the structure of the observations series (if there is one). i.e.: calculate Euclidean distance between a group of vectors and their projection mapping on the reference eigenspace associated with another group of vectors. the difference reflects the variation of data structure of the current observation from its predecessor. Z 2 diff 2 U T H T 2 = U T *Z 2

9 SSA-based Score Function Classic algorithm of SSA 1st step: Embedding Form the trajectory matrix of the sampling sequence 2nd step: Singular Value Decomposition (SVD) Obtain EigenVectors U 3rd step: Eigentriple grouping 4th step: Diagonal averaging.

10 SSA-based Score Function Embedding Use structure of hankel matrix to build a trajectory matrix X (a group of vectors )which represents multi-dimension variables with multiple spans Example: Sequence length: Time series : 3 : variable variable variable variable observation observation observation

11 SSA-based Score Function Singular Value Decomposition (SVD) Operation Use Base Matrix X for constructing lag-covariance matrix Output of SVD (R) U: Eigenvectors of S: Eigenvalues V: Eigenvectors of A subset of U forms the eigenspace for accepting mapping, so as to study structure variation of continuous data stream Z 2 diff 2 U T H T 2 = U T *Z 2

12 SSA-based Score Function Connection between two groups of observations: Base Matrix X Vs. Target Matrix Z [M x?] [M x?] If U is eigenvectors of If H is eigenvectors of If Z and X share the same data structure [? x M] * [M x?] [? x M] * [M x?] H Z [? x M] * [M x?] [? x M] * [M x?] Z T Z 2 diff 2 U H=Z T *U U T H T 2 = U T *Z 2

13 Applying individual observations on time series for the operation of SSA-based score function the length of input Base sequence is: nn the row number of Base Matrix X is : mm the column number of Base Matrix X is : nn - mm + 1 the length of input Target sequence is: mm-1+q-p the row number of Target matrix Z is : mm the column number of Target matrix Z is : q - p *. Setting proper parameters for desired performance is a challenge mm mm nn-mm+1 q - p

14 Sliding-window detection scheme 0 => 1 => 1 2 1

15 Workflow of SSA-based score function EigenV: Eigenvector Eu.: Euclidean ν λν Z T U H=Z T *U If U is eigenvectors of If H is eigenvectors of H Z 2 diff 2 U T HT 2 = U T *Z 2

16 Case Study Use data from DDoS attack for test our approach

17 Case Study nn =30 mm =20 nn =30 mm =6 nn =30 mm =15

18 Case Study nn =120 mm =80 nn =120 mm =30 nn =120 mm =60

19 Future Work Correct existent problems Develop more sophisticated detection scheme to reduce false alarm Optimizing parameter setting Applying other real cases for practice

20 Conclusion Demonstrate examples of network attacks Develop SSA-based score function for changepoint detection Conduct related experiment for verification Discuss the feature work

21 Questions?