Microphone Identification using Higher-Order Statistics

Transcription

1 Microphone Identification using Higher-Order Statistics Hafiz Malik 1, and John W. Miller 1 1 Information Systems, Security, and Forensics Lab, Department of Electrical & Computer Engineering, University of Michigan - Dearborn, Dearborn, MI Correspondence should be addressed to Hafiz Malik (hafiz@umich.edu) ABSTRACT This paper presents a statistical framework for microphone identification using only digital audio recordings. To accomplish this task, the microphone-induced artifacts are modeled using a nonlinear function and then a statistical tool based on higher-order statistics is used to capture these artifacts. More specifically, polyspectral analysis is used to capture microphone-induced artifacts. Distance- and correlation-based similarity measures are used for automatic microphone identification. More specifically, distance between scale-invariant Hu moments of the bicoherence magnitude spectrum and cross-correlation between bicoherence phase spectra are used. The effectiveness of the proposed framework has been tested with 24 audio recordings captured using eight microphones during three recording sessions. Performance of the proposed scheme is evaluated using ambient noise recordings captured using eight microphones of four different types. 1. INTRODUCTION The use of digital media (audio, video, and images) as evidence in litigation and criminal justice proceedings has become the norm. It is, therefore, critical to verify the authenticity and integrity of digital media for its admissibility into a court of law. Integrity authentication of digital recording in the absence of digital watermarking, fingerprinting, etc. is a challenging task. The availability of powerful, sophisticated, and easy- to-use digital media manipulation tools has made the media authentication process more complicated and challenging. However, available state-of-the-art methods are limited in their scope. They are unable to perform critical tasks in authentication such as determining if the evidentiary recording was assembled by manipulating and splicing original recordings captured in different environments and using different devices. The reason is that existing methods do not provide solutions to key problems, such as, uniquely mapping a digital recording to the recording device, which is important in audio forensics. The goal of microphone forensics focuses on finding answers to the following questions: Does a microphone leave a fingerprint in the recording? If so, how can we extract it? Does the microphone signature depend on the underlying transducer technology? Is a unique mapping of a recording to the source possible? This paper focuses on finding answers to these questions. The approach used here to identify microphones is based on the fact that modern digital devices (e.g. cameras, printers, scanners, RF devices, etc.) introduce a unique patterns when used for capturing scenes, printing, scanning, or processing an RF signal, etc. Digital cameras can be identified based on their intrinsic sensor s noise pattern as suggested in [4] and [2]. Similarly, printers can be identified based on their grey-level co-occurrence features, as described in [3]. Following the same logical reasoning, microphones can also be identified based on their intrinsic characteristic. This approach is the major source of motivation behind this work. The objective here is to devise a novel method for microphone identification, so that it can be used as an authentic tool for identification, tamper detection, and audio forensics applications. For the last several years, researchers and scientists have been working to classify microphones using statistical learning [7, 6, 9]. For example, Kreatzer et. al. in [7] AES 46 TH INTERNATIONAL CONFERENCE, Denver, USA, 212 June

2 proposed a statistical learning based framework to classify microphones and acoustic environments using digital audio recordings. This work is based on the concept of tupel-verification presented in [8]. This approach first computes a feature vector from the audio samples and then uses a Naive Bayes classifier and K-means clustering for classifying microphones and environment. This method achieves the classification accuracy better than random guessing but too low to be applied to practical scenarios. Moreover, this approach is only focussed on the inter-device analysis. In other work described in [5], microphones are classified based on Fourier coefficients. These coefficients are computed from all those segments of an audio sample which only contain noise, and then these coefficients are added to get a Fourier coefficient histogram. This histogram is used as a global feature vector. It is shown in [5] that the classification of microphones is done with an accuracy of 93.5% in the best case scenario. One of the limitations of the existing state of the art, for microphone identification, is that they are unable to uniquely map an audio recording to the microphone used. The main objective of this work is to address the aforementioned limitation by developing a model driven approach using microphone induced artifacts, extract them from audio recordings, and use them for microphone identification and forensic analysis. To accomplish this, we first model microphone artifacts using a nonlinear function. Higher order statistics [1] based on third-order cumulants (bispectrum) is used to capture microphone artifacts. Distance and correlation-based similarity measures are used for automatic microphone identification. More specifically, the distance between scale-invariant Hu moments of the bicoherence magnitude spectrum and cross-correlation between bicoherence phase spectra are used for microphone identification. The rest of the paper is organized as follows. The mathematical modeling of microphone distortions is provided in Section 2.1 and microphone identification framework is discussed in Section 2.2. Details of the data collection, feature extraction, and performance evaluation are outlined in Section 3. Concluding remarks and future directions are discussed in Section MATHEMATICAL APPROACH 2.1. Mathematical Modeling of Microphone Distortions Microphones are complex electromechanical devices where interaction between mechanical, electromechanical, and electrical elements transforms sound energy into electrical signals. Any nonlinearity in these elements results in a distorted output. In general, the stiffness of the mechanical suspension and acoustical damping are the dominant causes of nonlinear distortion in most microphones. Microphone distortions can be classified into harmonic, intermodulation, and differencefrequency distortions. Harmonic distortion is the effect of nonlinearity on a pure tone excitation, causing harmonic components in the output. Intermodulation distortion is the effect of nonlinearity produced at the output from an excitation that is sum of a stronger high frequency and a weaker low frequency ω 1 components. The difference-frequency distortion is the effect of a nonlinearity produced at the output from an excitation of sinusoids of same amplitude.the intermodulation effect produces an output signal made of sums and differences of the input signals fundamental frequencies and their harmonics, that is, ± ω 1, ± 2ω 1, ± 3ω 1, etc. The difference-frequency distortion is the effect of a nonlinearity produced at the output from an excitation of sinusoids of same amplitude, that is, the differencefrequency ω combination component in the response of frequencies e.g. 2 ω 1, 2ω 1, 3ω 1 2, etc. Microphone response is generally characterized in terms of physical parameters of a microphone. Consider, for example, a condenser microphone with capsule capacitance C o, input resistance R, diaphragm displacement, δ, and supply voltage V o. Assuming multisinusoidal input signal x(t)= i A i sin(ω i t), i A i T/4,i [r,s,q], w i C o R 1, and 2δ i w i C o R 1, the Output y(t), by truncating fourth-order and beyond harmonic and intermodulation terms, can be expressed as [13], y(t) = V o δ i sin(ω i t)+ V o i + V o 2(RC o ) 2 i δi 3 ωi 2 RC o i sin(3ω i t) δ 2 i ω i sin(2ω i t) + (ω2 r ± ω s )δ r δ s V o sin((ω r ± ω s )t) ω r ω s RC o + (2ω r± ω s )δr 2 δ s V o 2ωr 2 ω s (RC o ) 2 sin((2ω r ± ω s )t) AES 46 TH INTERNATIONAL CONFERENCE, Denver, USA, 212 June Page 2 of 1

3 + (2ω r± ω s ± ω q )δ r δ s δ q V o 2ω r ω s ω q (RC o ) 2 Similar mathematical modeling of microphone distortions can also be obtained for other microphone types [14, 12, 11, 1], e.g., carbon, micromachined silicon condenser, optical-fiber Mach - Zehnder, etc. The microphone response given in Eq. (1), can be approximated using the following discrete time-invariant Hammerstein series model, y[n] = k g 1 [k]x[n k]+ g 2 [k]x 2 [n k] k = g 3 [k]x 3 [n k] (1) k Here, g i (τ),i = 1,2,3 characterize the linear, quadratic, and cubic response of the system. It can be observed from Eq. (1) that nonlinearity introduced by a microphone can be described with a simple point-wise operation which introduce higher-order correlations. The higher-order spectral analysis can be used to analyze the nonlinearity of a system operating under a random input. The main motivation behind considering higher-order statistics (known as cumulants, and their associated Fourier transform, also known as polyspectra) is that these methods not only reveal amplitude information about a process, but also reveal phase information. Correlation (resp. power spectrum) based methods are phase blind. In addition, cumulant-based methods are blind to any kind of a Gaussian process, whereas, correlation is not; hence, cumulant-based signal processing methods are capable of handling nonlinearity and non-gaussian processes more effectively than correlation-based methods [16]. To illustrate effectiveness of polyspectral analysis to identify nonlinearity, consider input signal, x[n]=c 1 cos[ω 1 n+φ 1 ]+c 2 cos[ n+φ 2 ]. (2) passing through a second-order nonlinear system, that is, y[n]=x[n]+αx[2] 2, where α. It can be shown that second-order nonlinearity introduces harmonics with strongly correlated phases and magnitudes. More specifically, y[n] will contain input frequencies ((ω 1,φ 1 ) and (,φ 2 )); harmonic distortion at (2ω 1,2φ 1 ) and (2,2φ 2 ); and intermodulation distortion at (ω 1 +,φ 1 + φ 2 ) and (ω 1,φ 1 φ 2 ). It sin((ω r ± ω s ± ω q )t) is worth mentioning that the considered quadratic nonlinearity results in phase correlations at the sum and/or difference frequencies, also referred to as quadratic phase coupling (QPC). The QPC can be used to detect quadratic nonlinearity in stochastic processes. To this end, the bispectrum, C x 3 (ω 1, ), (which is the Fourier transform of the third-order cumulant, c x 3 (τ 1,τ 2 ),) can be used to detect QPC, where C x 3 (ω 1, )= and τ 1 = τ 2 = c x 3 (τ 1,τ 2 )e j(τ 1ω 1 +τ 2 ), (3) c x 3(τ 1,τ 2 )=cum[x[n]x[n+τ 1 ]x[n+τ 2 ]] (4) A sufficient condition for the existence of the bispectrum, i.e., is assumed here. τ 1 = τ 2 = c x 3(τ 1,τ 2 ) <, (5) Specific observations can be made regarding how a nonlinearity affects the bicoherence spectrum. Consider a pair of frequencies ω 1 and and the nonlinearity induced harmonic ω 1 ± whose magnitude is correlated to ω 1 and, which will result in a high magnitude value in the bicoherence magnitude. In addition, if the input frequencies have phases, φ 1 and φ 2, then the phase of the nonlinearity induced intermodulation component ω 1 ± is φ 1 ± φ 2. It is easy to see that the bicoherence has a zero phase and a bias towards π/2 may also occur due to harmonic auto-correlations. In general, the average bicoherence magnitude would increase as the amount of QPC grows. It can be concluded that a nonlinearity will contribute in the bicoherence in two ways: 1) an increase in the magnitude for certain harmonics, and 2) a phase bias towards and/or π/2 for nonlinearity induced harmonics. To illustrate this fact bicoherence magnitude and phase plots of the first (top), second (middle), and third (bottom) recordings of ambient noise captured using Behringer EMC8 measurement microphones (M 5 and M 6 ) are shown in Figs. 1 and Microphone Identification It can be observed from Fig. 1 that bicoherence magnitude plots exhibit 12 regions of symmetry [16]. The frequency region bounded by ω 1, and is considered for further analysis. As mentioned earlier in the AES 46 TH INTERNATIONAL CONFERENCE, Denver, USA, 212 June Page 3 of 1

4 M 5 : Behringer EMC8 1 Bicoherence Magnitude Plots M 6 : Behringer EMC ω ω 1 Fig. 1: Shown is the bicoherence magnitude plots of the first (top), second (middle), and third (bottom) recordings of ambient noise captured using two Behringer EMC8 measurement microphones (M 5 and M 6 ). M 5 : Behringer EMC8 1 Bicoherence Phase Plots M 6 : Behringer EMC ω ω 1 Fig. 2: Shown is the bicoherence plots plots of the first (top), second (middle), and third (bottom) recordings of ambient noise captured using two Behringer EMC8 measurement microphones (M 5 and M 6 ). previous Section 2.1 that nonlinearity effects both the magnitude and phase of the bicoherence. It has been observed that each microphone leaves characteristic artifacts in the resulting recording that can be observed via bicoherence magnitude and phase spectra. For microphone identification purpose both attributes of the bicoherence are therefore considered. To capture microphone dependent magnitude artifacts, scale-invariant moments are considered here. More specifically, scale invariant Hu moments [15] of the bicoherence magnitude spectrum are used to capture nonlinear correlations. Invariant moments (e.g., the Hu invariant set) are statistical measures designed to remain constant after some transformations, such as object rotation, scaling, translation, or image illumination changes. Therefore, the invariant set of Hu moments is a good candidate for robust and reliable pattern recognition applications. To this end, scale-invariant Hu moments are computed from the bicoherence magnitude image obtained from the test audio recording. The scale-invariant Hu moments µ i, j where i+ j 2 can be computed as, µ i, j = η i, j ( η 1+ i+ j 2 here, the central moment η i, j can be computed as, η i, j = x, ) (6) (x x) i (y ȳ) j b(x,y) (7) y where x and ȳ are the components of the centroid of bicoherence magnitude spectrum, b(, ). AES 46 TH INTERNATIONAL CONFERENCE, Denver, USA, 212 June Page 4 of 1

5 Fig. 3: Shown are the plots of interclass cross-correlation between the phase spectrum of the first recording of measurement microphone M 6 and the phase spectra of the first recordings made using three pairs of microphones, i.e., M i : i={1,2,3,4,7,8}. The correlation-based similarity measure is used to capture microphone dependent phase artifacts. A strong correlation between bicoherence phase spectra for the same microphone type (intraclass) and lack of it for the interclass microphones is the motivation behind considering correlation based similarity measure for microphone identification. To illustrate this fact, interclass cross-correlation between the phase spectrum of the first recording made using measurement microphone M 6 and the phase spectrum of the first recordings captured using the remaining six microphones M i : i={1,2,3,4,7,8} is shown in Fig. 3. Similarly, intraclass cross-correlation between the phase spectrum of the first recording made using measurement microphone M 6 and the phase spectrum of the second and third recordings using the same microphone, and the first, second, and third recordings using the second measurement microphone M 5 is shown in Fig. 4. It can be observed from Figs. 3 and 4 that the bicoherence phase spectra of recordings captured using M 6 is strongly correlated with the second and third recordings made using M 6. The weighted sum of the correlation-based similarity score between the bicoherence phase spectra of the two recordings and the Euclidean distance between the first four scale-invariant Hu moments (, µ 2,, µ 2,1, ) of the bicoherence magnitude spectrum can be used for microphone identification. 3. EXPERIMENTAL RESULTS AES 46 TH INTERNATIONAL CONFERENCE, Denver, USA, 212 June Page 5 of 1

6 Fig. 4: Shown are the plots of intra-class cross-correlations between the phase spectrum of the first recording made using M 6 and the phase spectra of the second and third recordings made using the same pair of microphones (M 6 & M 5 ). To test effectiveness of the proposed method, 24 audio recordings of ambient noise were captured using eight microphones. Details of the recording setup and data set used for performance evaluation are provided next Test Data Collection Setup and Test Data Recordings were made using eight microphones and an eight-channel USB multitrack recorder. The multitrack recorder, a Zoom R16, was powered via the USB connection and recordings were saved on a PC running Windows 7. A 12-inch fan approximately.7 meter from the microphones operating at maximum speed was used as the sound source. Details of the four pairs of microphones used are provided in Table 1 The microphones were directed towards the back of the fan so as to minimize any effects from wind noise. Three sets of recordings were made at a level of approximately 45 db for each channel with a duration of about one minute and about ten minutes between recordings. Phantom power was used on the channels for the EMC-8 microphones while the Optimus microphones had internal batteries. The recording software used was n-track version with the ASIO4ALL V2.1 driver Results Table 1: List of the makes and models of the microphones used. Mic. ID M 1 M 2 M 3 M 4 M 5 M 6 M 7 M 8 Microphone Details Samson R19 Dynamic Mic.-1 Samson R19 Dynamic Mic.-2 Radio Shack Electret Mic.-1 Radio Shack Electret Mic.-2 Behringer EMC8 Measurement Mic.-1 Behringer EMC8 Measurement Mic.-2 Zoom R16 Recorder Built-in Mic.-Left Zoom R16 Recorder Built-in Mic.-Right The effectiveness of the proposed method was tested on the data set recorded using four pairs of microphones. To this end, each recording is segmented into frames of four seconds duration with a 5% overlapping factor. Bicoherence is estimated from each audio segment using the direct (fft-based) approach [16]. The bicoherence is estimated with the following parameter settings: 1) 128- point segment length, 2) 256-point FFT length, 3) no overlap, and 4) Rao-Gabr optimal window for frequency domain smoothing. AES 46 TH INTERNATIONAL CONFERENCE, Denver, USA, 212 June Page 6 of 1

7 For each audio segment of a given recording the first four scale-invariant Hu moments, that is,, µ 2,, µ 2,1, and are computed from the bicoherence magnitude spectrum. Shown in Fig. 5 are the scatter plots of scaleinvariant Hu moments, µ 2,, and computed from the bicoherence magnitude spectra of the first (top), second (middle), and third (bottom) recordings made using a pair of Samson R19 dynamic microphones M 1 and M 2. Similarly, shown in Figs. 6 8 are the scatter plots of scale-invariant Hu moments, µ 2,, and computed from the bicoherence magnitude spectra of the first (top), second (middle), and third (bottom) recordings made using microphone pairs M 3 and M 4, M 5 and M 6, and M 7 and M 8, respectively. It can be observed from Figs. 5 8 that there are significant inter- as well as intraclass variations in the frame-based scale-invariant Hu moments. The interclass variations can be used for microphone type identification, whereas, intraclass variation can be used to achieve individual microphone identification. To illustrate, this scatter plots of average scale-invariant Hu moments, µ 2,, and µ 2,1 computed from the bicoherence magnitude spectra of the first, second, and third recordings using all eight microphones are shown in Fig. 9. Shown in Fig. 9 are scatter plots of average scaleinvariant Hu moments, µ 2,, and computed from bicoherence magnitude spectra of first, second, and third recordings made using microphone pairs M 1 and M 2, M 4 and M 4, M 5 and M 6, and M 7 and M 8. The first- and higher-order statistics of the estimated Hu moments are used for microphone identification. To this end, mean, variance, skewness, and kurtosis of the estimated frame-based Hu moments are used for microphone identification. Threshold based multiple hypothesis testing is used for microphone identification which resulted 1% correct classification of 24 recordings for eight classes (microphones). 4. CONCLUSION In this paper, the objective is to classify microphones based on microphone induced artifacts are modeled using nonlinear function. A statistical method based on bicoherence (third-order comulants) is then used to capture microphone induced nonlinearity. A similarity measure based on fourth and lower order statistics of frame-based scale-invariant Hu moments is used for microphone classification. Simulation results show that this approach can successfully identify an individual microphone using small intraclass and large interclass similarity. The results of this work have direct applications for forgery detection in digital multimedia recordings, speaker recognition systems, and sensor identification. The effectiveness of the proposed system is tested using ambient noise recording only. Currently, we are working on extending this approach to speech recordings. We are also working to investigate uniqueness of the non-linearity introduced by a microphone. 5. REFERENCES [1] H. Farid, Detecting Digital Forgeries Using Bispectral Analysis, Technical Report, AIM-1657, MIT AI Memo, [2] A.E. Dirik, H.T. Sencar, and N. Memon, Digital Single Lens Reflex Camera Identification From Traces of Sensor Dust, IEEE Trans. on Information Forensics and Security, 28, vol. 3(3), pp [3] A.K. Mikkilineni and P.-J. Chiang and G.N. Ali, Printer identification based on graylevel cooccurrence features for security and forensic applications, Proceedings of SPIE Intl. Conf. on Security, Steganography, and Watermarking of Multimedia Contents, 25, pp [4] J. Lukas, J. Fridrich, and M. Goljan, Detecting digital image forgeries using sensor pattern noise, in SPIE Electronic Imaging, Photonics West, 26. [5] R. Buchholz, C. Kraetzer, and J. Dittmann, Microphone Classification Using Fourier Coefficients, Lecture Notes in Computer Science, Springer Berlin/Heidelberg, vol. 586/29, 21, ISBN [6] C. Kraetzer, M. Schott, and J. Dittmann, Unweighted Fusion in Microphone Forensics using a Decision Tree and Linear Logistic Regression Models, Proc. ACM Multimedia and Security Workshop, 29, 29, pp [7] C. Kraetzer, A. Oermann, J. Dittmann, and A. Lang, Digital audio forensics: a first practical evaluation on microphone and environment classification, Proceedings of the 9th workshop on Multimedia and Security, 27. AES 46 TH INTERNATIONAL CONFERENCE, Denver, USA, 212 June Page 7 of 1

8 M 1 : Samson R19 1 M 2 : Samson R19 2 µ 2, µ 2, Fig. 5: Shown are scatter plots of scale-invariant Hu moments, µ 2,, and computed from the bicoherence magnitude spectra of the first (top), second (middle), and third (bottom) recordings made using a pair of Samson R19 dynamic microphones M 1 and M 2. [8] A. Oermann, A. Lang, and J. Dittmann, Verifiertuple for audio-forensic to determine speaker environment, Proc. ACM Multimedia and Security Workshop 25, 25. [9] D. Garcia-Romero, and C. Espy-Wilson, Speech forensics: Automatic acquisition device identification, J. Acoust. Soc. Am., vol. 127(3), 211, pp [1] M.T. Abuelma atti, armonic and intermodulation performance of optical-fiber Mach-Zehnder microphone, Applied Acoustics, vol 72(5), 211, pp [14] M.T. Abuelma atti, Harmonic distortion in electret microphones, Applied Acoustics, vol 34(1), 1991, pp [15] M.K. Hu, Visual pattern recognition by moment invariants, IRE Trans. Info. Theory, vol. IT-8, 1962, pp [16] C.L. Nikias and A. P. Petropulu, Higher-Order Spectral Analysis: A Nonlinear Signal Processing Framework Upper Saddle River, NJ: Prentice- Hall, [11] M.T. Abuelma atti, Large signal performance of micromachined silicon condenser microphones, Applied Acoustics, vol 68(6), 27, pp [12] M.T. Abuelma atti, Large signal performance of micromachined silicon inductive microphones, Applied Acoustics, vol 68(1), 27, pp [13] M.T. Abuelma atti, Improved analysis of the electrically manifested distortions of condenser microphones, Applied Acoustics, vol 64(5), 23, pp AES 46 TH INTERNATIONAL CONFERENCE, Denver, USA, 212 June Page 8 of 1

9 M 3 : Radio Shack Electret 1 Scatter Plot of Scale Invariant Hu Moments M 4 : Radio Shack Electret 2 µ 2, µ 2, Fig. 6: Shown are scatter plots of scale-invariant Hu moments, µ 2,, and computed from the bicoherence magnitude spectra of the first (top), second (middle), and third (bottom) recordings made using a pair of Radio Shack electret microphones M 3 and M 4. M 5 : Behringer EMC8 1 M 6 : Behringer EMC8 2 µ 3. µ 2, µ 2, Fig. 7: Shown are scatter plots of scale-invariant Hu moments, µ 2,, and computed from the bicoherence magnitude spectra of the first (top), second (middle), and third (bottom) recordings made using a pair of measurement microphones M 5 and M 6. AES 46 TH INTERNATIONAL CONFERENCE, Denver, USA, 212 June Page 9 of 1

10 M 7 : Zoom R16 Builtin Mic 1 M 8 : Zoom R16 Builtin Mic 2 µ 2, µ 1,2 µ 2, µ 1,2 Fig. 8: Shown are scatter plots of scale-invariant Hu moments, µ 2,, and computed from the bicoherence magnitude spectra of the first (top), second (middle), and third (bottom) recordings made using a pair of Zoom R16 built-in electret microphones M 7 and M 8. Scatter Plot of Scale Invariant Hu Moments x 1 5 Samson R19 1 Samson R19 2 RS Electret 1 RS Electret 2 Behringer EMC8 1 Behringer EMC8 2 Zoom R16 Built in 1 Zoom R16 Built in µ 2, Fig. 9: Shown is scatter plots of average scale-invariant Hu moments, µ 2,, and computed from the bicoherence magnitude spectra of first, second, and third recordings made using all eight microphones. AES 46 TH INTERNATIONAL CONFERENCE, Denver, USA, 212 June Page 1 of 1