Image Data Compression. Dirty-paper codes Alexey Pak, Lehrstuhl für Interak<ve Echtzeitsysteme, Fakultät für Informa<k, KIT

Size: px

Start display at page:

Download "Image Data Compression. Dirty-paper codes Alexey Pak, Lehrstuhl für Interak<ve Echtzeitsysteme, Fakultät für Informa<k, KIT"

Darleen Sullivan
6 years ago
Views:

1 Image Data Compression Dirty-paper codes 1

Reminder: watermarking with side informa8on Watermark embedder Noise n Input message m Auxiliary informa<on

c wn Watermark detector Watermark decoder Auxiliary informa<on Output message m n Blind embedding (no red

if WM is present, re-compute the embedded mark w a, find correla<on with c wn Watermarking with side

distor<on power P a If cover work is known a priori (or trivial), then scheme equivalent to AWGN channel In

as R I(X;Y ), where I(X; Y) is the mutual informa<on between the channel input variable X and the received

2 Reminder: watermarking with side informa8on Watermark embedder Noise n Input message m Auxiliary informa<on Watermark encoder! w a Cover work +! c o! c w Channel! c wn Watermark detector Watermark decoder Auxiliary informa<on Output message m n Blind embedding (no red arrow): To embed a mul<-bit message, need error-correc<ng code (trellis code / Viterbi algorithm) To find if WM is present, re-compute the embedded mark w a, find correla<on with c wn Watermarking with side informa6on (with red arrow): Limited channel rate R, given constraints on fidelity P e and channel distor<on power P a If cover work is known a priori (or trivial), then scheme equivalent to AWGN channel In that case, classical result by Shannon applies: R 1 2 log " 1+ P % e $ ' # P a & More generally, formulated as R I(X;Y ), where I(X; Y) is the mutual informa<on between the channel input variable X and the received signal Y. For jointly Gaussian X and Y, it is: I(X;Y ) = I(Y; X) = 1 2 log " E[X 2 ]E[Y 2 ] % $ ' # E[X 2 ]E[Y 2 ] E[XY ] 2 & Let us now consider some non-trivial cover work 2

3 Dirty-paper codes (DPC) [Costa 83]: imagine a sheet of paper covered with independent dirt spots of normally distributed intensity. In some sense, the problem of wri<ng a message on this sheet is analogous to that of sending informa<on through the channel. The writer knows the loca<on and intensity of the dirt spots, but the reader cannot dis<nguish them from the ink marks applied by the writer. Usual codes: one message corresponds to one codeword Dirty-paper codes: one message m è codebook C m (i.e. DPC = collec<on of codes) Given a message, the embedder selects the codeword that best fits the cover work The detector finds the most likely codeword from the union of all sub-codebooks, and outputs its syndrome i.e. the corresponding message Example: LSB embedding into a single numeric value Cover work: some real number n Embedded signal: a binary digit, m = {0, 1} Method: output n rounded to the nearest odd (m = 0) or even (m = 1) value 3

Least significant bit (LSB) watermarking Modify every work s value so that its least significant bit encodes a binary message: Original work Message Watermarked work = Each pixel value: v = 0 255 (8

actually changed LSB embedding is fragile against channel noise or global re-scaling Scalar quan<za<on: each pixel value is independently watermarked (i.e. the channel rate may be significantly lower than with the n-dimensional ICS quan<za<on) LSB effect may be visible in the uniform areas (e.

4 Least significant bit (LSB) watermarking Modify every work s value so that its least significant bit encodes a binary message: Original work Message Watermarked work = Each pixel value: v = (8 bits) Each value: m = 0 or m = 1 Watermarked value: v m = (v & ~1) m Changes each value by at most 1 unit (out of 255), difference is usually impercep<ble On average, only 50% of bit values are actually changed LSB embedding is fragile against channel noise or global re-scaling Scalar quan<za<on: each pixel value is independently watermarked (i.e. the channel rate may be significantly lower than with the n-dimensional ICS quan<za<on) LSB effect may be visible in the uniform areas (e.g., sky) Widely used for content authen6ca6on and steganography! Why use only one LSB? Why assume independent pixels? 4

5 Watermarking Gaussian signals naïve approach Let Z be a Gaussian random variable, jointly normal and uncorrelated with S (cover work) Let E[Z 2 ] = P e and X = S + Z be the watermarked signal Let n be sufficiently large. For each message m, choose a random codebook C m over Z with sufficiently many elements, so that the sub-codebook rate necessarily sa<sfies R(C m ) log 2 C m I(S; X) = 1 n 2 log E[X 2 ] E# $ Z 2 % & Following the proof of Shannon s theorem, for every signal s over S, there is a codeword x in C m such that d(s, x) < E[Z 2 ] = P e. In other words, sub-codebooks are sufficiently dense. Moreover, for large n and R(C m ) = I(S; X), the expected distor<on is E[d(s, x)] = E[Z 2 ] = P e. Finally, let C be the union of all C m. To transmit a message m for a given cover work s, the sender embeds a codeword x of C m that is the closest to s (it can always be found within the distance P e from s). The channel noise is Gaussian with power E[N 2 ] = P a, the received signal is Y = X + N. For unique decoding, density of union codebook must be sufficiently low: R(C) I(X;Y ) Finally, each message is encoded by mul<ple codewords. Thus, the transmission rate boundary is: R 1 I(X;Y ) I(S; X) = 1 2 log $ P a + P s P e ' & ) % P a P s ( (!) Random codebook constructed over variable X = S + Z Result depends on the sta<s<cs of works Rate always below the Shannon s limit (= for P s = 0) Rate approaches 0 as P s increases to Can we do beder than that? 5

6 Costa s theorem Before: codebook was constructed over X = S + Z. Re-write: X = (αs + Z) + (1 - α)s Let us now construct the codebook over the new variable U α = αs + Z, 0 < α < 1. As in the previous scheme, the received signal is Y = S + N + Z. By exactly the same reasoning as before, we obtain: R R α = I(U α ;Y ) I(S;U α ) Now compute the pieces: From the # of codewords in the union codebook # of codewords in sub-codebook I(S;U α ) = I(S;αS + Z) = 1 2 log " E[S 2 ]E[(αS + Z) 2 ] % $ ' = 1 # E[S 2 ]E[(αS + Z) 2 ] E[S(αS + Z)] 2 & 2 log " P s (α 2 P s ) % $ P s (α 2 P s ) α 2 2 ' = 1 # P s & 2 log " α 2 P s % $ ', # P e & I(Y;U α ) = I(S + Z + N;αS + Z) = 1 2 log " (P a + P s )(α 2 P s ) % $ ' = 1 #(P a + P s )(α 2 P s ) (αp s ) 2 & 2 log " (P a + P s )(α 2 P s ) % $ ', # P a (P e +α 2 P s )+ P s P e (1 α 2 )& R α = 1 2 log " P e (P s + P a ) % $ ' # P a (P e +α 2 P s )+ P s P e (1 α 2 )& Variables S, Z, N are independent, uncorrelated Find the op6mum parameter α to maximize the rate: α* = argmax R α = α P e P a, R α* = 1 2 log 1+ P e P a Scheme known as Distor<on- Compensated Quan<za<on (DCQ) Generaliza6on of regular quan6za6on Ideal Costa scheme (ICS): Max rate exactly as without (or with a known) cover work! (i.e. cover work has no influence on transmission rate) 6

7 LSB: quan8za8on index modula8on (= dither modula8on) Scalar quan6za6on according to uniform sub-codebooks: m = 0,..., M 1 C 0 Δ 0 M Δ 2M Δ C m = {(m + km )Δ, k Z} C 1 C 2 C = C m = { kδ, k Z} Δ (M + 2) Δ m C Embed message m into sample value s: find the nearest value x m in the sub-codebook C m "# Equivalent to quan<za<on: s m = Q MΔ;m/M (s), with quan<zer Q A;d (s) = s / A d $%+ d A Power of quan<za<on error: P e = ( s Q A;d (s)) 2 = A2, in case of QIM: P e = M 2 Δ " Received signal y, syndrome detec<on: m = y % # $ Δ& ' mod M If channel noise power is less than Δ2 /12, then the WM system is error-free Distor6on-compensated QIM, DCQIM (= scalar Costa scheme, SCS): Fix value α, quan<ze only frac<on α in sample s: s m = Q MΔ;m/M (αs)+ (1 α)s Hard to find op<mal α for SCS analy<cally! = α 1 Q [Eggers et al, 2003]: experimental op<mum (max rate): MΔ/α;m/M (s)+ (1 α)s * α SCS P e P e P a Expect: ICS performs bewer (has higher rate) than SCS, and SCS is bewer than QIM 7

8 Performance of scalar watermarking schemes SS = spread spectrum; blind embedding, work as Gaussian noise [Eggers et al, 2003] DWR = P s /P e Dither modula<on; binary DM = LSB = Rate NB: systema<c gap due to 1D-quan<za<on vs true nd-vector quan<za<on Watermark-to-noise ra<o, = P e /P a 8

LaPce codes Costa s theorem assumes that dimensionality (message length) n -> In prac<cal embedders, break work into blocks, embed short message (n ~ 1) in each block Want embedding/detec<on speed of

9 LaPce codes Costa s theorem assumes that dimensionality (message length) n -> In prac<cal embedders, break work into blocks, embed short message (n ~ 1) in each block Want embedding/detec<on speed of scalar scheme, but bewer rate Fundamental tradeoff between robustness and encoding cost: Code separa<on: need wide spacing between codewords Coset forma<on: need codewords of different messages to be closely spaced Dirty-paper code must allow distor<on compensa<on to balance between the two Direct binning (sample-based quan<za<on): efficient but too expensive Laqce (= linear) dirty-paper codes Dirty-paper code: (C, {C 0, C 1,, C M - 1 }) Each C i (and C) is linear wrt Z: c 1, c 2 C a c 1 + b c 2 C a, b Z C is generated by finite number of elements Each C i is a subset of C Example: Quincunx code Codebooks: (C, {C 0, C 1 }) Rate = 0.5 (encodes 2 messages) Non-separable, linear (2D vector quan<zer) Two cosets: red and green nodes Quincunx pawern 9

LaPce types Orthogonal la{ces: set on n orthogonal vectors rotated and shi ed scalar quan<za<on Non-orthogonal la{ces: any set of n non-coplanar vectors (rank of base matrix is n) Important proper6es

10 LaPce types Orthogonal la{ces: set on n orthogonal vectors rotated and shi ed scalar quan<za<on Non-orthogonal la{ces: any set of n non-coplanar vectors (rank of base matrix is n) Important proper6es of laqces: Quan<za<on error: ( ) 2 P e =! x Q Λ (! x) Voronoi cell: all points that are closer to the given node than to the other nodes Sphere packing density: ra<o of volume of one sphere to the volume of Voronoi cell Kissing number: how many spheres touch a given sphere # of deep holes: points with max distance from nodes inside one primary cell Orthogonal Z 2 la{ce Hexagonal A 2 la{ce Deep holes: by shi ing la{ce to DH, obtain another coset Used in prac<ce Dimension Best P e Z A 2 A 3 D 4 D 5 E 6 E 7 E 8 Dense packing Z A 2 A 3 D 4 D 5 E 6 E 7 E 8 Largest kissing number Z A 2 D 3 D 4 D 5 E 6 E 7 E 8 10

11 Prac8cal encoding with E 8 lapce E 8 laqce: E 8 = D 8 U (D * (1,1,1,1,1,1,1,1)), where D 8 = (subset of Z n (x x 8 ) mod 2 = 0) 15 deep holes => can build dirty-paper code with 16 messages (4 bits) WM embedding steps: 1. Trellis-encode incoming message x (error-correc<on), obtain embedded message x* 2. Divide x* into N blocks of 4 bits, divide work into at least N blocks 3. Correlate each block with 2 reference pawerns w 1 and w 2 => vector v of length 2N 4. Using E 8 la{ce and some distor<on compensa<on parameter α, embed 4 bits of x* into corresponding 8 vector coordinates 5. Modify each block of work to embed WM WM detec6on steps: 1. Project image w n onto 2N-dimensional vector v r, as in embedder 2. For every 8 dimensions in v r, awempt to quan<ze with all possible 4-bit messages 3. Quan<zer that results in smallest distor<on indicates the trellis-coded message 4. Apply Viterbi decoder to extract the embedded message x d Message errors (in %) Orthogonal la{ce E8 la{ce [Cox et al, 2009] Amount of noise (in standard devia<ons) 11

12 LaPce codes and valumetric scaling La{ce codes: simple, robust (AWGN), may approach full ICS rate of dirty-paper channel However: do not survive global re-scaling of work values! Possible solu6ons: Scaling-invariant marking space (ideally: real perceptual space) One op<on: embed WM into phase coefficients of Fourier-transformed work; Works well for images, does not work for audio signals. Ra<onal dither modula<on: adjust quan<za<on step based on some features of work Example: use some DCT coefficients (not used for watermarking) to determine the quan<za<on steps for the following coefficients (where WM is embedded); Can be used together with perceptual adapta<on (cf. next lecture). Invert scaling based on some separate embedded pilot signal Or determine scaling parameters from message-carrying WM (with enough data) Dirty-paper trellis codes: use the angles between codewords, rather than correla<ons Spherical codes: la{ce is on surface of unit sphere, corresponding distance func<on Modified trellis: addi<onal arcs, mul<ple paths represen<ng the same message Encoder: find op<mal path in a single coset (corresponding to the message being sent) Decoder: Viterbi algorithm, find best path in the union of all cosets. Problem: it is not so easy to generate good spherical codewords (random vectors do not work well for moderate dimensions) 12

13 Characteriza8on of errors in watermarking systems Message errors (incorrect decoding) Measure: bit error rate (BER) Protec<on: error-correc<ng codes False posi6ve (FP) errors: FP error rate / probability Depend on detector Random-WM FP (one of many WMs in a work) Random-work FP (single WM in many works) FN rate 3 1 Receiver opera<ng characteris<c (ROC) curve 2 False nega6ve (FN) errors: FN rate or probability Depend on embedder, channel and detector Related to effec<veness (no channel awack), robustness (normal processing distor<ons), security (hos<le awacks). Occurrence frequency No WM 2 3 FP rate WM ed works ROC curve: tradeoff between FP and FN errors When too liwle data exists to es<mate one or both axes, need interpola<on model (behavior near zero) 1 Detec<on value 13

14 Prac8cal watermarking: whitening filter Analysis of dirty-paper codes assumed that works are normally distributed (AWGN) We know that image pixels do have strong correla<ons (sta<s<cs of the 2 nd order) May use known correla<on matrix to perform PCA whitening before embedding/detec<on Otherwise, a poor-man s solu<on: z LC (! c,! w) z LC (! c! f,! w! f ) f whitening filter, * - convolu<on E.g.: delta-filter f = (-1, 1) removes horizontal correla<ons between the adjacent pixels If the correla<on matrix R of distribu<on is known, then f is the middle row of R -1 If the filter is computed for a wrong distribu<on, may instead introduce new correla<ons Unwanted effects most prominent in tails (low probability regions) Whitening has opposite effect on random-wm and random-work errors: works and WM codewords are usually drawn from different distribu<ons with different correla<ons Ozen used in prac6cal watermarking schemes! 14

Image Data Compression. Steganography and steganalysis Alexey Pak, PhD, Lehrstuhl für Interak;ve Echtzeitsysteme, Fakultät für Informa;k, KIT

Image Data Compression. Steganography and steganalysis Alexey Pak, PhD, Lehrstuhl für Interak;ve Echtzeitsysteme, Fakultät für Informa;k, KIT Image Data Compression Steganography and steganalysis 1 Stenography vs watermarking Watermarking: impercep'bly altering a Work to embed a message about that Work Steganography: undetectably altering a