ENGINEERING TRIPOS PART IIB: Technical Milestone Report

Size: px

Start display at page:

Download "ENGINEERING TRIPOS PART IIB: Technical Milestone Report"

MargaretMargaret Woods
5 years ago
Views:

1 ENGINEERING TRIPOS PART IIB: Technical Milestone Report Statistical enhancement of multichannel audio from transcription turntables Yinhong Liu Supervisor: Prof. Simon Godsill 1 Abstract This milestone report has introduced one fairly well developed click -noise restoration method for multi-channel audio data, using bi-variate Gaussian model with fixed noise covariance matrix. Performance and improvement directions of this method are explained in details. In future plan, a new iterative method that includes a time-varying noise variance is shown along with other future potential improvements. 2 Motivation Dust particles and scratches in the grooves of gramophone discs could lead to different types of degradations. Clicks is the term used to refer short burst of interference random in time and amplitude, which is a localized type of degradation. The clicks occurs frequent in most analogue disc recording with durations of clicks ranges from less than 20µs up to 4ms. In most examples at least 90% of samples remain undegraded, so there is potential to achieve convincing restoration. There has been fairly well developed foundation on click restoration for single-channel audio data1. This project is focusing on multi-channel audio click restoration, which has advantages due to more potential information available to achieve better interpolation results45. 1

2 3 Restoration Methods 3.1 Modeling of clicks The signal convention is inherited from 1: the noises are treated as additive noises. Therefore the localised degradation can be expressed as: y t = x t + i t n t with y t = ylt y rt representing the corrupted 2-channel signals, x t = xt x t is the underlying audio signal and i t is a 0/1 switching process, which would have value 1 if the corresponding signal point is corrupted. n t = nlt n rt effects are different on signals from two channels. is a corrupting noise process. The noise 3.2 Modelling of the underlaying signal We first partition the underlaying signal x into two parts, uncorrupted part x k and corrupted part x u in the following way: x = Ux u + Kx k U and K are rearrangement matrices. In the following research, the signal data x is assumed to be a random zero mean Gaussian vector with autocorrelation matrix R x = Exx T : P(x = N ( x 0, R x The covariance matrix would be Toeplitz if x is Wide-sense stationary. Therefore, We split the signal data into short segments with length of 5000 samples each, which is around 0.11s when sampling frequency is 44.1kHz. There could be other model for the underlaying signal, for example autoregressive (AR model3. The performance comparison between different models may be investigated in the future. 3.3 Modeling of the noise Although the noise effects on 2 channels are different, they are correlated with each other. We first treat the noise as time-independent Bivariate-Gaussian with a constant covariance matrix and tune the covariance matrix to give best interpolation performance. Throughout this section, We use the vector convention below: T yu1 L yu1 R yu2 L yu2 R and Nx u = y LR u y LR u = x u1 x u1 x u2 x u2 is a reshaped vector of all corrupted signal data. N is a rearrangement matrix which shapes x u into desired format. 2 T

3 3.3.1 MAP interpolation The noise covariance matrix has three parameters (σ1,σ and ρ and therefore: nlt N ( 0, Σ σ 2 where Σ = 1 ρσ 1 σ 2 n Rt ρσ 1 σ 2 σ2 2 For the full noise vector, it follows multi-variate Gaussian: n LR = y LR u Nx u = n L1 n R1 n L2 n R2 N (0, C m Σ where C m = Σ (1 The conditional probability for unknown samples given all known information could be expressed: P ( x u x k, y L, y R, σ 2 1, σ 2 2, ρ = P( x, y L, y R σ 2 1, σ 2 2, ρ P ( x k, y L, y R σ 2 1, σ 2 2, ρ = P ( ( x P nlr C m P ( x k, y L, y R σ1, 2 σ2, 2 ρ The denominator is a constant. Therefore, the MAP interpolation for the unknown samples could be given by maximizing the nominator with respect to the unknown samples: x u N ( x 0, R x N ( nlr 0, C m = 0 which is equivalent to: x T R 1 x x + (yu LR Nx u T C 1 m (yu LR Nx u x u Therefore, the MAP interpolation is: = 0 x MAP u = ( M u + N T C m 1 N 1( N T C m 1 y LR u M k x k (2 where M u = U T R 1 x U and M k = U T R 1 x K is equivalent to extracting the elements of the matrix which correspond to unknown and known samples Performance and Improvements From the interpolation results below, most click spikes with different amplitudes are successfully restored. Although there are still some marks remained for some clicks, the audio becomes clearer in terms of both plots and actual sounds. It could be noticed that there are a few very small clicks remained unchanged. This is because the click detection mechanism is still primary, as the project is focusing on click restoration. 3

Figure 1: Example segment 1 Figure 2: Example segment 2 For further improvement directions, there are a few points worth mentioning: The auto-correlation matrix R x is calculated based

4 Figure 1: Example segment 1 Figure 2: Example segment 2 For further improvement directions, there are a few points worth mentioning: The auto-correlation matrix R x is calculated based on the corrupted data. The parameters for noise covariance matrix may not be tuned to the best. Clicks could have time-varying covariance matrix, but now Σ is constant for all noise. 4

5 A new interpolation method that could overcome the aforementioned problems is under developing and will be introduced in the future plan section. 4 Future Plan 4.1 Noise with time-varying covariance matrix As explained above, due to the randomness in click s magnitudes, a fixed noise covariance matrix for all cases may not be suitable. Therefore we consider combining likelihood and prior to give time-variable covariance matrix Σ t. The prior P(Σ t is assumed to follow inverse Wishart distribution W 1 (Ψ, ν. Ψ should have the same dimension as Σ t and ν is a scalar. For following calculation, we have vector convention as below: C m = Σ 1 Σ 2 Adding to this, the way we calculate R x = Ey L yl T above is based on corrupted samples. To improve this, we repeat the interpolation process and update ˆx u, ˆR x and Ĉ m for each iteration2. Steps for this iterative method: 1. Initialize the noise covariance matrix with an initial guess. Note the initial guess should have same structure as C m in (1, so directly using fixed covariance matrix C m is a good choice. ˆR x = Ey L yl T. Initialize the auto-correlation matrix with corrupted data 2. ˆx u = arg max x i P ( x u x k, y L, y R, ˆR x, Ĉ m 3. ˆΣ t = arg max Σ t P ( Σ t n t Ĉ m = ˆΣ 1 ˆΣ 2 4. ˆR x autocorrelation(ux u + Kx k 5. Loop back to step 2 for a certain times. Step 2 could be calculated directly by (2, step 4 is straightforward and in step 3, as we assume Σ t is independent over time, we could find the MAP for each Σ t to construct the 5

6 covariance matrix C m : P(Σ t n t P(n t Σ t P(Σ t = N (n t 0, Σ t W 1 (Σ t Ψ, ν = W 1 (Σ t A + Ψ, n + ν (3 Where A = n t n t T which is 1. Therefore we could have: and n is the number of data pair used to calculated the likelihood, P(Σ t n t (n + ν + p log Σ t tr ( (A + ΨΣ 1 t Where p p is the dimension of Σ t. The Maximum of this expression is: ˆΣ t = 4.2 Performance Quantification A + Ψ (n + ν + p It is essential to develop a method to quantify restoration performance, so that results from different methods could be compared. Noisy lead-in parts of turntable discs should be added to a clean audio record (e.g. Record from CD. They could be used as the target data and different restoration methods are applied to them. By calculating the mean square error between the restored and clean data, the performance could be quantified. 4.3 Forward and Reverse recording As the clicks have prolonged tails after turntable needles meet defects and dusts on the discs, the corrupted data would be different if the discs are playing reversely. Therefore, more information is available and better restoration is expected. References 1 Simon J. Godsill and Peter J. W. Rayner; Digital Audio Restoration; 1998 Springer; Edition. ed. edition. 2 Simon J. Godsill; EM for the multi-channel Gaussian NNMF model; April 15, H Lin, S Godsill The multi-channel AR model for real-time audio restoration; 2005, IEEE. 4 Saeed V. Vaseghi; Advanced Digital Signal Processing and Noise Reduction; 2008, John Wiley & Sons Ltd. 5 Klaus Linhard; Noise-reduction method for noise-affected voice channels 6

Continuous State MRF s

EE64 Digital Image Processing II: Purdue University VISE - December 4, Continuous State MRF s Topics to be covered: Quadratic functions Non-Convex functions Continuous MAP estimation Convex functions EE64