Imperfect Stegosystems

Transcription

1 DISSERTATION DEFENSE Imperfect Stegosystems Asymptotic Laws and Near-Optimal Practical Constructions Tomá² Filler Dept. of Electrical and Computer Engineering SUNY Binghamton, New York April 1, 2011

2 2 of 29 Steganography / Steganalysis Steganography is a mode of covert communication. Message Message Message encryption Shared encryption/decryption keys Shared stego key Message decryption Cover image Embedding function Stego image Steganalyzer Extraction function Cover source Alice Eve Bob GOAL: embed messages so that the source of stego images is statistically undistinguishable from the source of cover images.

3 3 of 29 Why is steganography relevant? First ocially reported case: June 2010 FBI reported that Russian spy gang hid secret information in images which they posted on the Internet. Method was broken by revealing the secret keys. Spies were deported. Other known cases in history: message tattooed on slave's head - ancient Greeks microdots - Germans in WW2 Steganalysis is even more important for law enforcement agencies but cannot be advanced without advancing steganography.

4 4 of 29 Security in Steganography Security is dened in terms of statistical distinguishability. Steganalysis: Statistical test between cover/stego H 0 : given image is cover H 1 : given image is stego Perfectly secure stegosystem (Cachin): Cover distribution P and stego distribution Q satisfy D KL (P Q) = P(x) P(x) log x Q(x) = 0 Implies P = Q every detector makes random guessing. Stegosystem is imperfect (ε-secure) if D KL (P Q) = ε > 0.

5 5 of 29 Imperfect Stegosystems Imperfect stegosystem: cover and stego distributions are dierent statistical detectors exist. perfectly secure stegosystems exist for articial cover sources all known stegosystems for digital media are imperfect digital media cover sources will hardly ever be perfectly understood to allow perfectly secure stegosystems It is important to study imperfect stegosystems for complex cover sources.

6 6 of 29 Dissertation Outline Gibbs Sampler Markov Chains MI Embedd. Square- Root Law Gibbs Random Fields Fisher Inform. How does secure payload scale with number of pixels? Multi- Layered Constr. Imperfect Stegosystems: Asymptotic Laws & Practical Constructions Gibbs Construction Syndrome- Trellis Codes Viterbi Alg. Distortion Design Convol. Codes SVMs How to design and implement practical embedding schemes? Steg. Features

7 Part I IMPERFECT STEGOSYSTEMS: ASYMPTOTIC LAWS How does the secure payload scale with the number of cover elements?

8 8 of 29 Asymptotic Laws for Secure Payload Secure payload: Number of bits one can hide in n-element cover while reaching certain level of detectability by the passive warden. Perfectly secure stegosystems: Assuming full knowledge of the cover source, the secure payload of perfectly secure stegosystems scales linearly in n. Communication rate of perfectly secure stegosystems is positive. Imperfect stegosystems: How does secure payload scale for imperfect stegosystems? Many hints suggest that secure payload is sublinear in n.

9 9 of 29 Mutually Independent Embedding Operation Impact of embedding is modeled as probabilistic mapping acting on each cover element independently - MI embedding. Pr(Y l = j X l = i) = b ij (β) X l... l-th cover element Y l... l-th stego element β... change rate (rel. payload) Matrix B = (b ij ) is transition probability matrix for β 0. LSB embedding: B = = I + βc = + β = 1 β = β = 1 = 1

10 10 of 29 Perfectly Secure Cover Source Cover source is perfectly secure w.r.t. given MI embedding the resulting stegosystem is perfectly secure.

11 11 of 29 Perfectly Secure Covers w.r.t. MI embedding Invariant distributions of MI embedding: Matrix B is stochastic has k 1 left eigenvectors (invariant distributions) π (a), a {1,...,k} to 1, π (a) B = π (a).

12 11 of 29 Perfectly Secure Covers w.r.t. MI embedding Invariant distributions of MI embedding: Matrix B is stochastic has k 1 left eigenvectors (invariant distributions) π (a), a {1,...,k} to 1, π (a) B = π (a). Example (perfectly secure cover): If P(X 1 = i, X 2 = j) = π (a), then P is perfectly secure. i π (a ) i

13 11 of 29 Perfectly Secure Covers w.r.t. MI embedding Invariant distributions of MI embedding: Matrix B is stochastic has k 1 left eigenvectors (invariant distributions) π (a), a {1,...,k} to 1, π (a) B = π (a). Example (perfectly secure cover): If P(X 1 = i, X 2 = j) = π (a), then P is perfectly secure. i π (a ) i Elements distributed independently with some invariant distribution form perfectly secure cover source. Set of all perfectly secure distributions form convex hull. There are exactly k n linearly independent perfectly secure cover sources on n elements.

14 12 of 29 Assumptions Summary 1 COVER SOURCE - Markov Chain rst order Markov Chain with tran. prob. mat. A = (a ij ) 2 EMBEDDING ALGORITHM - MI embedding independent substitution of states 3 STEGOSYSTEM - ε-secure (imperfect) stegosystem is IMPERFECT (ε-secure) stego cover i i j k l a ij a jk akl... j k l HMC embedding MC

15 Square Root Law of Markov Covers Under the assumptions of Markov covers, MI embedding, and imperfect stegosystem, we prove the following theorem. Theorem (SRL for Markov Covers): 1 embedding payload that grows slower than n leads to eventual ε-security for arbitrarily small ε > 0 2 embedding payload that grows exactly as A n leads to ε-secure stegosystem with xed ε > 0 3 embedding payload that grows faster than n leads to eventual detection with arbitrary P FA and P MD This implies that the secure payload of imperfect stegosystems with Markov covers and MI embedding scales as A n. BEST PAPER AWARD of 29

16 14 of 29 Fisher Information Rate From Taylor expansion... (nβ = # of changes) ( D KL P (n) Q (n) β ) = 1 nβ 2 I + O(β 3 ) = ε 2 I... Fisher information (rate) at β = 0. Secure payload of ε-secure stegosystems scales as r n. Root rate: (more rened measure of capacity) r 1 I I can be expressed as a quadratic form w.r.t. C and f (A).

17 15 of 29 Convex Combination of LSB and ±1 Embedding Use ±1 embedding in rst λ n pixels and LSB embedding in the rest. r(λ) 2 1 NRCS NRCS-JPEG CAMRAW Root rate r(λ): c λ = λc ±1 + (1 λ)c LSB r(λ) = 1 c T λ Fc λ λ 0 - LSB 1 - ±1 Higher root rate = better method.

18 Part II IMPERFECT STEGOSYSTEMS: NEAR-OPTIMAL PRACTICAL CONSTRUCTIONS From steganography to statistical physics Motivated by B O S S Break Our Steganographic System Break Our Steganographic System

19 17 of 29 Gibbs Construction For FIXED cover x stego y Y is distributed according to π. General detectability measure: D(x, y) D(x,y) is a cost of changing cover x to stego y. Assume D(x,y) < for all y Y and existence of D. What distortion measures are interesting? D(x,y) = F (x) F (y)... HUGO [IH 2010] F ( )... feature vector used in blind steganalysis D(x, y) can be derived from some side information

20 17 of 29 Gibbs Construction For FIXED cover x stego y Y is distributed according to π. General detectability measure: D(x, y) D(x,y) is a cost of changing cover x to stego y. Assume D(x,y) < for all y Y and existence of D. What distortion measures are interesting? D(x,y) = F (x) F (y)... HUGO [IH 2010] F ( )... feature vector used in blind steganalysis D(x, y) can be derived from some side information PAYLOAD-LIMITED SENDER: choose π such that entropy H(π) = m bits and avg dist. E π [D(x,y)] is minimal. DISTORTION-LIMITED SENDER: choose π such that average dist. E π [D(x,y)] = D ε and entropy H(π) is maximal.

21 18 of 29 The Separation Principle 3 basic problems to solve. (1) What distribution π solves the embedding problems? PLS: min π E π [D(x,y)] subject to H(π) = m. DLS: max π H(π) subject to E π [D(x,y)] = D ε.

22 18 of 29 The Separation Principle 3 basic problems to solve. (1) What distribution π solves the embedding problems? PLS: min π E π [D(x,y)] subject to H(π) = m. DLS: max π H(π) subject to E π [D(x,y)] = D ε. (2) How to simulate the best possible embedding algorithm? Algorithm based on D can be tested by blind steganalysis without being able to embed real message in practice.

23 18 of 29 The Separation Principle 3 basic problems to solve. (1) What distribution π solves the embedding problems? PLS: min π E π [D(x,y)] subject to H(π) = m. DLS: max π H(π) subject to E π [D(x,y)] = D ε. (2) How to simulate the best possible embedding algorithm? Algorithm based on D can be tested by blind steganalysis without being able to embed real message in practice. (3) How to implement the embedding algorithm in practice? Implement practical algorithm with payload close to H(π) and average distortion close to E π [D(x,y)].

24 19 of 29 (1) Optimality of Gibbs Random Field Let D(x,y) < for all y Y be a detectability measure. Optimal distribution π solving either PLS: min π E π [D(x,y)] subject to H(π) = m, or DLS: max π H(π) subject to E π [D(x,y)] = D ε has the form of a Gibbs Random Field π(y) = 1 Z(λ) exp ( λ D(x,y) ), ( ) Z(λ) = exp λ D(x,y) y Y λ 0 chosen to satisfy emb. constraint (PLS/DLS). Z(λ)... partition function (sum of Y terms HUGE). since D(x,y) <, then π(y) > 0 for all y Y.

25 20 of 29 (2) Sampling from the Gibbs Random Field Sampling from the GRF is well known and studied problem in statistical physics. One such algorithm is Gibbs sampler (Markov-Chain Monte Carlo): [Geman&Geman 1984] (1) Start with an arbitrary image, for example x. (2) Apply series of sweeps. In every sweep, all pixels are updated according to local statistics obtained so far. Properties of the GS algorithm y k... sample obtained after k sweeps (1) y π (allows to sample from π) (2) 1 k k D(x,yk ) E π [D] (allows to calculate expected val.) (3) H(y k y k 1 ) H(π) (not necessarily capacity achieving)

26 (3) Practical Embedding Algorithm Embedding algorithm is equivalent to a coset quantizer. Pixel-additive distortion function: D(x, y) = i ρ i (x, y i ) Syndrome-Trellis Codes: (SPIE 2010, WIFS 2010, TIFS 2011) message is communicated as a syndrome of a convolutional code represented in a dual domain embedding (nding the closest stego) is solved using Viterbi algorithm. Fast and very ecient solution. m x costs{ρ i } BEST PAPER AWARD 2010 STC encoder y = arg min D(x, y ) Hy =m y STC decoder Hy H = 0 0 m 21 of 29

27 22 of 29 Application to Spatial-Domain Digital Images undetectable 0.5 Average error of SVM-based steganalyzer with SPAM features coding loss BOWS2 database Payload-limited sender simulated emb. STC/ML-STC pentary (±2) 0.1 binary (±1) LSB matching alg. ternary (±1) ternary binary detectable Relative payload α (bits per pixel)

28 23 of 29 Distortion Optimization Using Blind Steganalysis 1 dene class of distortion functions with parameter θ 2 nd θ that maximizes steganalyzer error How to evaluate detectability of a given embedding method? 1 embed chosen payload into a large database of images 2 extract features from cover & stego images 3 train machine-learning based classier (Gaussian SVM) 4 report chosen error metric, such as trustworthy measure 1 P E = min P FA 2 (P FA + P MD ) very slow (requires many images, slow to train classier)

29 24 of 29 Detectability Criterion: Margin of Linear SVM decision boundary gi = 1 w gi = +1 support vectors misclassied samples SVM training problem: 1 min w R d 2 wt w + C ξ (w;f i, g i ) i margin regularization term

30 Detectability Criterion: Margin of Linear SVM decision boundary gi = 1 w gi = +1 L2R_L2LOSS detectability criterion: support vectors misclassied samples SVM training problem: 1 min w R d 2 wt w + C ξ (w;f i, g i ) i margin regularization term 1 embed chosen payload in N images and extract features 2 L2R_L2LOSS = size of the margin from soft-margin L 2 -regularized L 2 -loss linear SVM We used N = 80 images to evaluate L2R_L2LOSS. Takes 1 2 seconds on 40CPU cluster for images. 24 of 29

31 Optimized w.r.t. CC-PEV features and tested with CDF set. 25 of 29 Application to DCT-Domain Digital Images undetectable 0.5 Average error PE Gaussian SVM Simulated embedding Inter-block optimized non-shrinkage F5 0.1 Real impl. STC h = 10 Inter-block optimized detectable Relative payload α (bits per non-zero AC coefficients)

32 26 of 29 Conclusion Gibbs Sampler Motivated by B O S S Ö ÇÙÖ ËØ ÒÓ Ö Ô ËÝ Ø Ñ Ö ÇÙÖ ËØ ÒÓ Ö Ô ËÝ Ø Ñ Markov Chains MI Embedd. Square- Root Law Gibbs Random Fields Fisher Inform. How does secure payload scale with number of pixels? Multi- Layered Constr. Imperfect Stegosystems: Asymptotic Laws & Practical Constructions Gibbs Construction Syndrome- Trellis Codes Viterbi Alg. Distortion Design Convol. Codes SVMs How to design and implement practical embedding schemes? Steg. Features

33 List of Published Papers 1/2 Square-Root Law: [1] The Square Root Law of Steganographic Capacity for Markov Covers, with A. D. Ker, J. Fridrich, Proc. SPIE, [2] Complete Characterization of Perfectly Secure Stegosystems with Mutually Independent Embedding Operation, with J. Fridrich, IEEE ICASSP [3] Capacity of ε-secure Steganography, with J. Fridrich, Information Hiding, Syndrome coding: [4] Binary Quantization using Belief Propagation with Decimation over Factor Graphs of LDGM Codes, with J. Fridrich, Proc. Allerton Conference [5] Wet ZZW Construction for Steganography, with J. Fidrich, IEEE, WIFS [6] Minimizing Embedding Impact in Steganography using Trellis-Coded Quantization, with J. Judas and J. Fridrich, SPIE [7] Minimizing Additive Distortion Functions With Non-Binary Embedding Operation in Steganography, with J. Fridrich, IEEE WIFS, [8] Minimizing Additive Distortion in Steganography using Syndrome-Trellis Codes, with J. Judas, J. Fridrich, IEEE T-IFS Steganography: [9] Design of Adaptive Steganographic Schemes for Digital Images, with J. Fridrich, SPIE, [10] Using High-Dimensional Image Models to Perform Highly Undetectable Steganography, with T. Pevny and P. Bas, Information Hiding, [11] Steganography Using Gibbs Random Fields, with J. Fridrich, ACM MMSec, [12] Gibbs Construction in Steganography, with J. Fridrich, IEEE T-IFS 2010.

34 List of Published Papers 2/2 Digital forensics: [13] Managing Large Database of Camera Fingerprints, with J. Fridrich, M. Goljan, SPIE [14] Camera Identication - Large Scale Test, with J. Fridrich, M. Goljan, SPIE, [15] Using Sensor Pattern Noise for Camera Model Identication, with J. Fridrich, M. Goljan, IEEE, ICIP Reports: [16] Important properties of normalized KL divergence under the HMC model, [17] Capacity of ε-secure Steganography-proofs, Best paper awards: [1] Digital Watermarking AlIiance - SPIE 2009 The Square Root Law of Steganographic Capacity for Markov Covers. [2] Digital Watermarking AlIiance - SPIE 2010 Minimizing Embedding Impact in Steganography using Trellis-Coded Quantization. BEST PAPER AWARD 2009 BEST PAPER AWARD 2010

35 We can only see a short distance ahead, but we can see plenty there that needs to be done. ALAN TURING, (1950)