An Information-Theoretic Analysis of Dirty Paper Coding for Informed Audio Watermarking

Transcription

1 1 An Information-Theoretic Analysis of Dirty Paper Coding for Informed Audio Watermarking Andrea Abrardo, Mauro Barni, Andrea Gorrieri, Gianluigi Ferrari Department of Information Engineering and Mathematical Sciences, University of Siena, Italy & CNIT Research Unit of Siena Department of Information Engineering, University of Parma, Italy & CNIT Research Unit of Parma Abstract Upon the introduction a simplified distortion model for a Gaussian audio watermaking scenario, we derive a lower bound for the Gelfand-Pinsker capacity of the watermark channel. Then, we use the capacity bound for designing an efficient practical watermarking scheme based on dirty trellis codes. The proposed information-theoretic framework is then validated, through the use of experimentally acquired audio signals, with the use of a recently proposed frequency-domain audio watermarking. Both an ideal (reflection-free) Gaussian acoustic channel and a realistic multipath acoustic channel are considered. I. INTRODUCTION Digital watermarking is nowadays a rather well-understood field, backed by a solid theoretical background. However, the application of theoretical findings to real systems is often problematic due to the deviation of practical working conditions from the assumptions underlying the theoretical analysis. The most striking example is the difficulty of applying the informed watermarking paradigm [1], derived by modeling the watermarking problem as one of channel coding with side information at the encoder [2], to certain practical situations, in which the oldest spread-spectrum paradigm is often preferred by system designers. This is often the case, for instance, in audio watermarking systems designed to couple high imperceptibility with robustness against heavy distortions including temporal de-synchronization, D/A and A/D conversions, environment noise addition, multipath and microphone distortion. It is a matter of fact that the best performing schemes proposed so far to address this scenario are not based on informed watermarking. Spread-spectrum schemes are easy to implement, are very flexible, and can be applied to different signal representation domains, e.g., frequency and time domains. The most undesirable characteristic of spread spectrum watermarking is that the host signal acts as an interference and, therefore, must be treated as a disturbing noise limiting the watermark channel capacity. As opposed to spread spectrum watermarking, watermarking theory shows that, under certain hypotheses, it is possible to completely eliminate the interference between the watermark and the host signal, thus reaching the same performance obtained by systems where the decoder can access the original non-watermarked signal [1], [3], [4]. To achieve this goal, it is necessary to resort to so-called informed watermarking schemes, also known as dirty paper coding schemes, like the well known Quantization Index Modulation (QIM) approach [1]. At least in some application scenarios, then, there is an evident hiatus between theory and practice. While it is not easy to understand which are the conditions whose deviation from the theoretical assumptions determine the difficulty of fulfilling the expectations promised by informed watermarking, psychoacoustic considerations surely play a major role. In audio watermarking, the inaudibility of the embedded signal is usually obtained by relying on a masking threshold which determines the maximum allowed distortion. In particular, a psychoacoustic mask defines the maximum modification which can applied to each frequency sample. This represents a major deviation from classical distortion models in which inaudibility is defined in terms of Mean Square Error (MSE), thus making the direct application of the dirty coding paradigm, derived from a theoretical analysis, problematic. Additionally, when the energy of the noise, added after watermark embedding, is comparable to the energy of the host signal, the advantage ensured by informed watermarking decreases. In this paper, we first propose a theoretical approach that, under proper approximations, allows to compute the capacity of the Dirty Trellis Codes and Spread Spectrum (DTC-SS) system proposed in [5]. Then, by comparing the theoretical findings with some simulation results, we show how the theoretical approach can predict when and how the informed watermarking paradigm performs better than its non-informed counterpart. The paper is organized as follows. In Section II, the watermark transmission model is presented, together with the underlying assumptions. In Section III, the computation of the capacity of the informed watermarking system is described. In Section IV, we compare the theoretical findings with the performance results, relative to experimentally acquired audio signals, of the DTC-SS digital watermarking system proposed in [5]. In particular, two propagation environments are considered: an AWGN channel (Subsection IV-A) and a more realistic multipath channel (Subsection IV-B). Finally, Section V concludes the work. II. CHANNEL AND TRANSMISSION MODEL In the following, we denote by S the interference signal representing the features to be marked. Hence, we assume that

2 2 the transmitted signal is forced to take binary antipodal values which determine the following fixed error: X = ±. (1) The considered scenario is referred to as Dirty Paper Channel with Fixed Distortion (DPC-FD) and it accurately models the audio watermarking scenario considered in [5], provided that: (i) S represents the magnitude (in logarithmic scale, db) of the Modulated Complex Lapped Transform (MCLT) samples extracted from the audio signal; (ii) the admissible distortion is fixed. The latter assumption is justified by the fact that the system model presented in [5] is such that, for each MCLT samples, the allowed modification is limited by a maximum acceptable value, namely γ max, which, in practice, is often reached. A. Problem Formulation III. THE CAPACITY OF A DPC-FD Costa s expression for the capacity of a DPC is a particular case of the Gelfand-Pinsker capacity of a channel with random parameters non-causally known at the transmitter [6]. In particular, the channel output can be written as follows: Y = X +S +Z where S is the underlying Gaussian interfering signal (the host signal to be marked), X is the transmitted signal (the watermark), and Z is an independent Gaussian random variables with variancen, which represents the noise attack. In general, the Gelfand-Pinsker capacity can be expressed as [6]: C GP = max {I(U;Y) I(U;S)} (2) p(u,x s) where the maximization is performed over all possible joint probability density functions (pdfs) of the channel input X and of the auxiliary random variable U conditioned on the channel status S. Random binning is strictly related to the presence, in (2), of the auxiliary random variable U and to the concept of typical sequences. As shown in [6], the capacity can be achieved by generating a code C containing 2 n(i(u;y) ε) independent and identically distributed (i.i.d.) sequences drawn according to the pdf of U and splitting them into a number of bins, each containing2 n(i(u;s)+δ) sequences, where ε and δ are arbitrarily small positive quantities. Given a message (hereafter indicated as a sequence of bits w) and a sequence of channel states s, the transmitter looks into the bin indexed by w for a sequence that is jointly typical 1 with s according to p(u s), let us call it u(s,w). Then, it transmits a sequence x which is jointly typical with s and u(s,w). Upon receiving a sequence y, the decoder looks for a sequence in C which is jointly typical with y according to p(y u) and outputs the index of the bin the decoded sequence belongs to. Given that the number of bins is approximately equal to 1 Roughly speaking two sequences are jointly typical if their joint empirical pdf is in accordance with the joint pdf of the sources that emitted them see [7] for a rigorous introduction to the use of typical sequences in information theory. 2 n(i(u;y) I(U;S)), any rate lower than C GP can be achieved. Costa showed that for the DPC channel, considering U = X +αs with α = P/(P+N) the capacity in (2) is maximized and this maximum is equal to the Shannon s capacity for the AWGN channel. Our goal is to redefine the maximization problem in (2) when Y = X +S +Z, as in Costa s model, but the alphabet of X is forced to be ±. For comparison purposes, we assume, as in Costa s paper, that S and Z are zero-mean independent Gaussian random variables with variances Q and N, respectively. Note that this is equivalent to assume that the interference signal S is a normalized (i.e., unitary mean) lognormal random variable. As for Z, while in the time domain Gaussian assumption is reasonable, in the considered watermarking scheme we have to consider the samples of noise in the logarithmic MCLT domain, so that Z is in general not Gaussian. However, our results (not shown here) confirm that approximating Z as Gaussian is reasonable. Our goal is then to solve the following maximization: C DPC FD = max p(u,x s) s.t. x X(s) {I(U;Y) I(U;S)} (3) where X(s) = {, }. Note that in the considered model the distortion power is fixed and equal to P = 2. Note that in (3) we have denoted the capacity C DPC FD to put in evidence the fact that it represents the capacity of the DPC-FD scenario considered in this paper. B. A Lower Bound for the Capacity First of all, observing that p(u,x s) = p(x u,s)p(u s), it can be concluded that the maximization in (2) reduces to finding the optimized expressions for p(u s) and p(x u, s). In order to find such expressions and solve the maximization problem (2), we impose the following relationship between the auxiliary random variable U and S: U = sign(ξ +αs) (4) where: sign(x) is the signum function; ξ is a Gaussian random variable independent from S, with zero mean and unitary variance; and α > 0 is a parameter whose value needs to be selected so that I(U;Y) I(U;S) is maximized. Of course, imposing the expression (4) limits our search space and, hence, a lower bound for C DPC FD will eventually be determined. The validity of this approximation will be proved later, by showing that, at least for moderate to low bit-rates, the lower bound approaches Costa s capacity. Taking into account (4), we now investigate p(u s) and p(x u,s). We first consider p(u s). Since, owing to its expression (4), U assumes the values 1 and 1 with the same probability, we focus, without loss of generality, on the probability that U = 1, conditioned to a certain value of S. It then follows that: A(s) Pr{U = 1 S = s} = 1 ( 2 erfc αs ) (5) 2

3 3 where erfc(x) = 2/ π x e t2 dt. As a next step, we need to determine p(x u,s). To do so, we invoke Proposition 1 in [6] and assume that X is a deterministic function of U and S. In particular, since x is forced to take binary antipodal values, a natural choice is to consider x = u. At this point, owing to the expressions of p(u s) and p(x u,s), the parameter α needs only to be determined. In order to solve (3), the optimized value of α is obtained by maximizing the objective function in (3), which, for ease of notational simplicity is defined as follows: R(α) I(U;Y) I(U;S). (6) R(α) WNR = 10 db WNR = 5 db The optimized value of α and the corresponding value of R in (6) (which corresponds to a lower bound for the capacity C DPC FD in (3)) will be denoted as α and R = R(α ), respectively. It is worth noting that for α = 0 the auxiliary random variable U is independent of S and A(s) = 0.5 regardless of the value of s. In this case, the proposed scheme is virtually identical to a classical Spread Spectrum (SS) embedding scheme. By recognizing the similarity of our scheme and a classical SS scheme, in the following we refer to proposed scheme as Informed-Coded SS (IC-SS). In order to evaluate R(α), let us consider first the quantity I(U;S). One has: I(U;S) = H(U) H(U S) = 1 h(a(s))p S (s)ds (7) where h(p) = plog 2 p (1 p)log 2 (1 p). In order to evaluate I(U;Y), we start by considering the conditional pdf P Y (y U = 1). One can write: P Y (y U = 1) = (a) = 2 = 2 P Y (y,s U = 1)ds P Y (y,s,u = 1)ds P Y (y s,u = 1)A(s)P S (s)ds (8) where in (a) we have exploited the fact that p(u = 1) = 1/2. Since x = u, we have: P Y (y U = 1) = 2 πn e (y s )2 2N A(s)P S (s)ds. (9) Since, owing to symmetry, it holds that P Y (y U = 1) = P Y ( y U = 1), the unconditioned pdf of Y can be expressed as follows: P Y (y) = 1 2 P Y(y U = 1)+ 1 2 P Y( y U = 1) α Fig. 1. R(α) as a function of α for two values of WNR, namely -10 db and -5 db. In both cases, DWR = 20 db. The following expression for I(U;Y) can then be derived: I(U;Y) = P Y (y U = 1)log 2 (P Y (y U = 1)) dy P Y (y U = 1)log 2 (P Y (y U = 1)) dy P Y (y)log 2 (P Y (y)) dy. (10) By taking into account expression (7) for I(U; S) and expression (10) for I(U;Y), it is now possible to evaluate R(α) in equation (6) by means of numerical integration. In particular, by evaluating R(α) for α [0,1], the value α, in correspondence to which R = R(α ) is maximized, can be determined. In Fig. 1, R(α) is shown as a function of α considering two possible value of the watermark-to-noise ratio WNR P/N, namely: -10 db and -5 db. In both cases, the state-to-signal (or document-to-watermark) ratio DWR Q/P is set 20 db. In both curves shown in Fig. 1, maxima can be clearly identified and the corresponding values α can be immediately determined. The same behavior can be observed for different values of γ, WNR, and DWR. C. Discussion In order to assess how close to capacity the lower bound derived in the previous section is, we have compared the maximum rate R achieved by the proposed IC-SS scheme with Costa s limit, which represents an upper bound for C DPC FD. Moreover, we have also compared the maximum rate R achieved by the proposed IC-SS scheme with the ultimate achievable rate of the Scalar Costa Scheme (SCS) evaluated in [8]. Note that, as discussed in [8], for low WNRs, the SCS can be interpreted as a distortion compensated-spread spectrum (DC-SS) scheme or, equivalently, as an improved spread spectrum (ISS) scheme with linear embedding. More

4 4 R (bits/coefficient) R (bits/coefficient) C (Costa scheme) SCS SS R * (IC SS) DWR = 20 db C (Costa scheme) SCS SS R * (IC SS) (a) DWR = 10 db (b) Fig. 2. Performance comparisons among Costa s capacity limit and the achievable rates of IC-SS, SCS, and SS schemes. Two possible values of DWR are considered: (a) 20 db and (b) 10 db. precisely, in [8] a typical knee behavior of the achievable rate of SCS is observed. In other words: below a critical value of WNR, the achievable rate is the same as those of the DC-SS scheme and of the ISS scheme with linear embedding; for values of WNR above the critical value, the behavior of the achievable rate of SCS follows that of Eggers et al. s scheme [9] with DWR =. For the purpose of comparison, we also evaluated R(0) (i.e., setting α = 0), as this corresponds to the achievable rate of a traditional SS scheme. The achievable rates obtained with the schemes outlined in the previous paragraph are shown in Fig. 2. In particular, two values of DWR are considered: (a) 20 db and (b) 10 db. As anticipated, the achievable rate of the proposed IC-SS scheme approaches Costa s capacity for low values of the, which correspond to moderate-to-low bit-rates. 2 In this regime, in fact, R = C, whereas SCS lies several dbs below 2 A payload of 0.1 bits per sample can be considered a moderate payload for most watermarking applications. the achievable rate. Eventually, the SS scheme achieves the same asymptotic performance of IC-SS when the bit rate tends to zero, but it is considerably worse than IC-SS for practical values of WNR. Finally, we observe that, as opposed to Costa s setup, the DWR has an impact on the achievable performance, since the range of bit-rates for which the lower bound becomes indistinguishable from C DPC depends on DWR. IV. PERFORMANCE ANALYSIS WITH EXPERIMENTALLY ACQUIRED AUDIO SIGNALS According to the approach outlined in Subsection III-B and as confirmed by the results in Fig. 1, for given values of WNR and DWR, the optimized value α, in correspondence to which the achievable rate R = R(α ) is maximized, can be determined. In particular, the corresponding values of I(U;Y) and I(U;S) can be computed and, from them, the corresponding optimal structure of the DTC-SS scheme presented in [5] can be determined as follows. We preliminary recall the informed watermarking scheme presented in [5], where trellis codes, in which the same information is represented by several paths of the trellis, are used. More specifically, we consider a trellis-based transmission scheme where only one out of M input bits represents the actual information bit, while the other M 1 bits, referred to as free bits, are used to create different paths in the trellis associated with the same information sequence. In this context, the set of trellis paths indexed by the free bits can be intepreted as the elements of a bin associated to a given information sequence, thus yielding a dirty trellis code. Hence, the overall gross transmitted rate of the auxiliary channel, comprising the free bits, is M times the net information rate. Drawing a parallel with the theoretical framework presented in Section III, where the rate of the auxiliary channel is I(U;Y) and the actual information rate is R(α) = I(U;Y) I(U;S), the optimal M can finally be determined as: M = I(U;Y) I(U;Y) I(U;S). (11) In the following, we investigate the validity of the predictions of the theoretical approach outlined in Section III considering two acoustic propagation channel models. First, we consider a simple AWGN channel: this is an idealized acoustic propagation channel with no secondary paths (i.e., reflections). Then, we consider a more realistic acoustic propagation channel with multipath. While the AWGN channel case is compliant with the assumptions of the theoretical framework, the multipath channel case is not. A. AWGN Channel In Fig. 3, the optimized value of M, derived from the theoretical model from R, is shown, as a function of the SN R = Q/N, for two single track (mono) audio clips, experimentally acquired with a sampling frequencyf s = Hz, that represent two different music styles: rock and jazz. It can be observed that, increasing the, the optimal value of M increases. For high s, one can take advantage from the knowledge of the host track; for low s, one have

5 jazz rock M=1.5 M=2 M=2.5 M=3 M= M 1.9 BER Fig. 3. M as a function of for 2 different tracks which represent two different music styles: rock and jazz. a lower advantage in the use of the informed scheme of [5], because the channel distortion is comparable with the host signal interference. In order to verify if the theoretical findings match the reality, we present the performance results obtained by applying the DTC-SS watermarking scheme [5], implemented in Matlab, to the experimental acquired audio signals. The watermark is inserted in the frequency interval [1400, 4400] Hz. In Fig. 4, the bit error rate (BER) of the receiver (where the watermark is extracted) is shown, as a function of the, for the two tracks considered in Fig. 3: (a) jazz, (b) rock. In each case, various values of M are considered, namely: 1, 1.5, 2, 2.5, 3. In all cases, γ max is set to 3 db. The following observations can be carried out from the results in Fig. 4. First of all, one can note that, for both jazz and rock audio tracks, for low s a good performance is reached when M = 1: this means that the embedded bits are completely independent from the host track and, hence, the system turns out to be a non-informed embedder with classical clean trellis encoding. Note that the DTC-SS system considered in [5] foreseen that higher values of M, i.e. higher dirtiness, is achieved at the expenses of a lower spreading gain. For low s the informed scheme brings no advantage, as the channel introduces a distortion much more serious than the interference related to the host signal, therefore, the use of M = 1, which maximizes the spreading gain, is the best approach. By comparing Fig. 3 with Fig. 4 (a) we can see that, for the jazz track, the theoretical findings are confirmed by experimental results. Fig. 3 shows that the optimized values of M lie between 1.55 (for low s) and 2.15 (for high s). Fig. 4 (a) shows that, when the is low, both M = 1.5 and M = 2 guarantee a good performance, while for higher s the best performance is obtained with M = 2. Increasing M further leads to a performance loss. Regarding Fig. 4 (b), we can note that for the rock track the theoretical results are not so accurate as in the case of the jazz track. In particular, the best performance is obtained with M = 1.5 while, according to the results in Fig. 3, the BER (a) (b) Fig. 4. BER, as a function of the, for the considered audio clips: (a) jazz and (b) rock. The considered propagation model is represented by a simple AWGN channel. In both cases, various values of M are considered. optimal value of M should lie between 1.7 and However, while the curve related to M = 2 is quite near to that of M = 1.5, the other curves (associated to the other values of M) show a more evident performance degradation. This slight misalignment between the theoretical results and the experimental ones is likely due to the assumption, made in Subsection III-A, that Z is a zero-mean Gaussian random variable. In fact, although this assumption might be verified in the time domain, in the considered watermarking scheme the noise samples belong to the logarithmic MCLT domain and this implies that Z can not be Gaussian. B. Realistic Multi-path Acoustic Channel In this subsection, we investigate the performance of the considered DTC-SS watermarking scheme using, for the propagation environment, the more realistic audio channel model presented in [5]. In Fig. 5, the BER is shown, as a function of the, for the jazz track, considering various values of M. First of all, it can be observed that, regardless of the value of M, the performance significantly degrades with respect to the AWGN case. Although the assumptions behind the theoretical framework do not apply to this propagation environment (Z M=1.5 M=2 M=2.5 M=3 M=1

6 6 BER M=1.5 M=2 M=2.5 M=3 M=1 [5] A. Abrardo, M. Barni, and G. Ferrari. Audio informed watermarking by means of dirty trellis codes. In Information Theory and Applications Workshop (ITA), pages 1 8, [6] S. I. Gelfand and M. S. Pinsker. Coding for channel with random parameters. Problems of Control and Information Theory, 9(1):19 31, [7] T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley Interscience, New York, [8] L. Pérez-Freire, F. Pérez-González, and S. Voloshynovskiy. An accurate analysis of scalar quantization-based data hiding. IEEE Trans. on Information Forensics and Security, 1(1):80 86, March [9] J. J. Eggers, R. Bäuml, R. Tzschoppe, and B. Girod. Scalar Costa scheme for information embedding. IEEE Trans. on Signal Processing, 51(4): , April Fig. 5. BER, as a function of the, for the jazz audio clip. The considered propagation model is represented by a multi-path audio propagation channel. Various values of M are considered. is not Gaussian at all), a value of M between 1.5. and 2.5 still guarantees the best performance. This means that, in this scenario as well, the informed approach outperforms the non-informed one, especially for high values of the. For low values, the noise energy is comparable to the host signal energy, so that the advantage brought by informed watermarking decreases. V. CONCLUSIONS In this work, we have proposed an information-theoretic modeling framework for the DTC-SS watermarking system introduced in [5]. In particular, we have first derived an upper bound (R ) on Gelfand-Pinsker capacity of a simple Gaussian watermark channel. Then, we have shown how this framework can be used to determine an optimized value of the parameter M, which quantifies the amount of dirtiness of the code (i.e., the amount of embedded information related to the host signal), of the DTC-SS algorithm in [5]. The obtained results, based on the use of experimentally acquired audio traces, show that the proposed information-theoretic framework allows to accurately predict optimized values of M in a simple AWGN communication scenario. Although the proposed theoretical approach is no longer valid for a realistic multipath acoustic channel, the obtained results show that in this propagation scenario as well the informed approach is still the best choice, especially for high values. REFERENCES [1] B. Chen and G. Wornell. Quantization index modulation: A class of provably good methods for digital watermarking and information embedding. IEEE Trans. Inform. Theory, 47(4): , May [2] I. J. Cox, M. L. Miller, and A. L. McKellips. Watermarking as communications with side information. Proc. IEEE, 87(7): , Jul [3] P. Moulin and J. A. O Sullivan. Information-theoretic analysis of information hiding. IEEE Trans. Inform. Theory, 49(3): , March [4] A. S. Cohen and A. Lapidoth. The Gaussian watermarking game. IEEE Trans. Inform. Theory, 48(6): , June 2002.