LEARNING DISCRIMINATIVE BASIS COEFFICIENTS FOR EIGENSPACE MLLR UNSUPERVISED ADAPTATION. Yajie Miao, Florian Metze, Alex Waibel

Transcription

1 LEARNING DISCRIMINATIVE BASIS COEFFICIENTS FOR EIGENSPACE MLLR UNSUPERVISED ADAPTATION Yajie Miao, Florian Metze, Alex Waibel Language Technologie Intitute, Carnegie Mellon Univerity, Pittburgh, PA, USA {yiao, fetze, ABSTRACT Eigenpace MLLR i effective for fat adaptation when the aount of adaptation data i liited, e.g., le than 5. The general otivation i to repreent the MLLR tranfor a a linear cobination of bai atrice. In thi paper, we preent a fraework to etiate a peaker-indepent dicriinative tranfor over the cobination coefficient. Thi dicriinative bai coefficient tranfor (DBCT) i learned by optiizing dicriinative criteria over all the training peaker. During recognition, the ML bai coefficient for each teting peaker are firtly found, on which DBCT i applied to give the final MLLR tranfor dicriination ability. Experient how that DBCT reult in conitent WER reduction in unupervied adaptation, copared with both tandard ML and dicriinatively trained tranfor. Index Ter Speaker adaptation, dicriinative training, peech recognition. INTRODUCTION Speaker adaptation i widely ued to build peakerdepent odel which can recognize peech fro unknown peaker. The ot coonly ued approach for peaker adaptation i axiu likelihood linear regreion (MLLR), which involve etiation of peaker-pecific linear tranfor over acoutic odel paraeter []. MLLR can perfor robutly given liited adaptation data. However, when the aount of adaptation data becoe really all, ay le than 5, peaker adaptation baed on MLLR doe not alway lead to iproved recognition perforance. Thi i becaue the etiation of MLLR tranfor i too noiy and doe not generalize well to the teting data. To olve thi proble, eigenpace-baed ethod have been propoed [2, 3, 4]. Generally, there are two tage in thi type of ethod. During the training tage, an appropriate et of bai atrice are coputed on the training data, uing MLfahion [4] or PCA-like algorith [2, 3]. During teting, the adaptation tranfor of a pecific peaker i repreented a a cobination of the bai atrice. Since the nuber of free paraeter, i.e., cobination coefficient, i reduced greatly, thee ethod can iprove the robutne of MLLR. Meanwhile, there ha been coniderable interet in exploiting dicriinative criteria for iproving MLLR adaptation. In the upervied ode, it ha been hown that dicriinative linear tranfor (DLT) can bring ignificant iproveent over ML tranfor [5, 6]. However, the gain of DLT drop draatically in unupervied adaptation becaue DLT i very enitive to uperviion error. A ore recent work i to learn a global dicriinative apping tranfor (DMT) on the training data, which can ap MLetiated adaptation tranfor to dicriinative tranfor [7, 8]. Since only ML etiation i perfored during adaptation, thi DMT ethod i found to be le enitive to hypothei error and thu ore uitable for unupervied peaker adaptation. In thi paper, we cobine thee two line of work and propoe a fraework which etiate a dicriinative linear tranfor over the bai coefficient in eigenpace MLLR adaptation. Thi global peaker-indepent tranfor, referred to a DBCT, act a a linear apping function fro ML-etiated bai coefficient to dicriinative one. During training, thi DBCT i learned by optiizing dicriinative criterion on the training peaker. During recognition, DBCT i ued to tranfor peaker-pecific bai coefficient etiated in a noral ML anner. The final adaptation atrix derived fro DBCT-tranfored coefficient becoe dicriinative in nature and at the ae tie robut to hypothei error. We ue the axiu utual inforation (MMI) criterion [9] for DBCT training and only exaine MLLR adaptation of HMM-GMM ean. However, thi approach can be extended eaily to the iniu phone error (MPE) criterion [9] and other for of adaptation tranfor uch a fmllr. Experient with Switchboard data how the effectivene of DBCT in iproving unupervied adaptation. 2. EIGENSPACE MLLR ADAPTATION The idea of eigenpace MLLR adaptation i to etiate MLLR tranfor in a ubpace contrained by the bai atrice. Thee bai atrice are learned on the training et

2 either with PCA or iterative MLE. Specifically, the MLLR tranfor W for peaker i repreented a N () ()() W dn W n () n where Wn repreent the n-th bai atrix. Speaker-pecific bai coefficient dn are etiated to optiize the ML objective on the adaptation data. If the feature dienion i D, the nuber of bai coefficient N i generally uch aller than D(D+), i.e., the ize of paraeter in the conventional MLLR adaptation. Therefore, thi type of eigenpace ethod perfor robutly under inadequate adaptation data. In principle, our DBCT approach can be ued with any eigenpace MLLR ethod. In thi paper, we deal with a pecific ipleentation derived fro the bai repreentation of fmllr decribed in [0]. The bai atrice are obtained via ingular value decopoition (SVD) on top of MLLR tatitic collected fro the training peaker, together with appropriate precon-ditioning. Totally we etiate D(D+) bai atrice which are orted by a decreaing order on their eigenvalue. For each teting peaker, an iterative line earch algorith i adopted to find the bai coefficient (equivalently the MLLR tranfor) which optiize the ML objective on the adaptation data. Copared with other, thi ipleentation ha the advantage that the nuber of bai atrice to be ued, i.e., N, can be decided dynaically according to the aount of available adaptation data. For exaple, N i et to the iniu of D(D+) and, where i the nuber of adaptation peech frae for thi peaker and i a contant uch a 0.2 [0]. In thi paper, we call thi ethod bai-mllr. Intereted reader can refer to [0] and the ipleentation of bai-fmllr in the Kaldi toolkit. 3. DISCRIMINATIVE BASIS COEFFICIENTS TRANSFORM Thi ection forally decribe how to learn the dicriinative DBCT over the bai coefficient. After uing bai-mllr on the training et, we can get the ML bai coefficient for each training peaker. Since bai atrice have been orted by their iportance [0], the firt P coefficient repreent the ot iportant one, i.e., λ [,,, ] (2) ()()() d d P which ha been extended with an additional. The reaining coefficient for the econd vector λ [,, ] (3) ()()() 2 dp dn () Applying DBCT, oted a W, to λ coefficient vector i ()(), (), the tranfored λ W λ (4) where W ha the ize of P(P+). Conidering DBCT on a all ubet of P coefficient ha two notable benefit. Firt, in teting adaptation, the actual coefficient dienion to be ued ay be uch le than D(D+). Thu, the DBCT tranfor odeled on the ot iportant dienion can till be applicable even under highly liited adaptation data. Second, if uing the whole et of coefficient, the affine DBCT ha the ize of [D(D+)+][D(D+)], which i expenive to anipulate. To facilitate the etiation of W, we iolate bai atrice fro coefficient by integrating bai atrice into GMM ean vector. For Gauian coponent, we have () the D P atrix M = [,, P ] and the D (N() () P) atrix M 2 = [ P, N () ], where the i-th () () colun vector i repreent the ean vector μ tranfored by the bai atrix W i, i.e., () () i = Wi ξ (5) () () where ξ i the extended vector of μ. Then, it natural () to derive the peaker-pecific ean μ, with the DBCT Wdct applied, a follow: ()()()() μ M W λ M λ (6) 2 2 Our goal i to obtain the peaker-indepent W which can optiize the dicriinative criterion on the training data. The tandard MMI optiization chee, baed on the weakene auxiliary function [9], i ued. The auxiliary function w.r.t. W i forulated a: Q() W,()() Wˆ x μ Σ x μ - +, t nu () T () t t t () T () t ()() xt μ Σ xt μ, t ()() () T D ()() μˆ μ Σ μˆ μ (7) nu where () t and () t are poterior occupancy of coponent being at tie t given the nuerator and oinator lattice, t i the peech frae of peaker, () D i a oothing ter with repect to peaker and coponent to enure the convergence of the dicriinative update. Following [8], we et thi ter to be D E () t where the contant E = 0.8. Alo, t () μˆ i the adapted ean which i calculated uing Eq. (6) and the current Ŵ, rather than the ean adapted by bai-mllr. The above DBCT can be etiated efficiently with an expectation axiization (EM) tyle algorith. For liit of

3 pace, we are oitting the detailed derivation. In the E tep, the following two type of peaker-pecific tatitic are collected for all the training peaker: G M Σ M ()()()() T ()()() T () T () t t t ()() + D T () T M μ λ K M Σ x λ Σ (8) nu where ()()() t t t, the accuulated coponent occupancy i D () t (9) and the two coponent pecific ter in can be calculated a () xt x t M2 λ2 ()()() μ μˆ M2 λ 2 (0) t K In the M tep, it can be proved that DBCT etiation ha the following updating forula to optiize the dicriinative auxiliary function in Eq. (7): ()()() vec() W vec kron G, P K () where the vec(.) operator tack the row of a atrix into a ingle vector, kron(.) i the Kronecker product of two atrice, P i the catter of the firt-part coefficient ()()() T vector defined in Eq. (2), that i, P λ λ. Given a well trained acoutic odel, for exaple dicriinatively trained HMM-GMM, iterative etiation procedure for DBCT are uarized a follow: () () Etiate the ML bai coefficient dl for each training peaker with bai-mllr. ()0 (2) Initialize W [ I; 0] and et k = 0. () k (3) Collect tatitic uing Eq. ( 8) and etiate W according to Eq. (). (4) k = k +. Go to tep 3 if not converged. After obtaining the DBCT, we can ue it in recognition. For a teting peaker, dicriinative adaptation with DBCT i perfored a follow: () Perfor firt-pa decoding to generate the uperviion hypothei for the utterance of peaker. () (2) Etiate ML bai coefficient dl with bai-mllr. (3) Adapt the acoutic odel paraeter uing W and () the ML coefficient dl according to Eq. (6). (4) Decode the teting et with the adapted odel. Fro the training proce, we can ee that DBCT depend on pecific acoutic odel. If the acoutic odel change, DBCT need to be re-etiated. 4. EXPERIMENTS The perforance of DBCT i evaluated on an Englih converational telephone peech tak. The training data contain 898 peaker (converation ide), around 72 hour, fro the Switchboard- corpu. The teting data coe fro a ubet of the 200 HUB5 evaluation et, coniting of 20 peaker and hour of peech. Acoutic odeling i baed on a 3-dienional MFCC front-end including the C0 energy and it firt, econd derivative with per-peaker ean noralization. An LDA tranfor reduce the feature dienion to 40, on which MLLT i applied. The ML odel ha 3000 clutered triphone tate, with an average of 2 Gauian per tate. The MMI criterion [9] i ued on top of the ML baeline and generate the peaker-indepent MMI odel. In MMI training, the nuerator lattice are built fro the reference, while the oinator lattice are produced by the ML odel with a heavily pruned unigra language odel. Our experient are conducted with thi MMI-SI odel, which ha a firt-pa WER of 38.5% on the teting et. During unupervied adaptation, bai-mllr etiation i perfored given the hypothei output fro MMI-SI. For all the decoding run, we ue a trigra language odel built only with the training trancription. 4.. Effectivene of DBCT A dicued in Section 3, DBCT i applied on a all ubet of the bai coefficient. Therefore, it can be robutly etiated with liited training data. To verify thi, we perfor DBCT learning on variou training et with different ize. Table preent the reult for the adapted MMI odel, uing tandard bai-mllr and bai-mllr +DBCT repectively. The coefficient dienion P, on which DBCT i applied, i et to 0. On each training et, we run DBCT etiation for 4 iteration and give the final MMI objective. Note that we are not reporting the actual MMI objective a coputed in [9]. Intead, the objective here equal the part in Eq. (7) depent on W. We can ee that uing DBCT in addition to bai-mllr adaptation yield further reduction on WER. Reducing the aount of training data reult in uperior recognition perforance and MMI objective. The bet WER i achieved on the 9-hour et, where DBCT bring 0.7% abolute iproveent to bai-mllr. With ore training data available, we oberve decreaed MMI objective, which indicate that DBCT ay not reach the optial point. Thu, for the 72-hour et, we run DBCT etiation for ore iteration and achieve the bet WER in the 6 th iteration. Depite a larger MMI objective, the recognition perforance i only 0.% better than uing 9 hour data. Thi confir that we are able to learn DBCT only with a all training et. If we hrink the training et further to 3 hour, the perforance of DBCT degrade ignificantly.

4 Table. Perforance (WER%) of bai-mllr with DBCT in unupervied adaptation. WER MMI Obj Bai-MLLR DBCT 72Hr DBCT 36Hr DBCT 8Hr DBCT 9Hr DBCT 3Hr DBCT 72Hr (6th iteration) 4.2. Senitivity to Superviion Error The following two ubection exaine propertie of DBCT. Unle tated otherwie, we how the reult of DBCT trained with 9 hour data. A a global tranfor, DBCT hould be le enitive to uperviion error copared with DLT [5, 6]. To invetigate thi point, three type of adaptation uperviion are ued. The baeline hypothei are fro MMI-SI in the firt-pa decoding. Hypothei of wore quality are generated by the ML odel. Finally, the correct reference i alo taken a uperviion. Thee uperviion are ued to etiate bai-mllr coefficient, on which DBCT i applied. During DLT training, the nuerator lattice are built fro thee uperviion and the oinator lattice are generated by MMI-SI. Table 2 preent the WER coparion with variou uperviion. For bai-mllr, uing the reference obtain.4% abolute gain over the ML-SI and MMI-SI uperviion. Thi i iilar to DBCT perforance difference. In contrat, for DLT, the reference ha 3.9% abolute iproveent over ML-SI and 3.5% over MMI-SI uperviion. Thi how that DBCT i le enitive to the quality of uperviion and thu uitable for unupervied adaptation. It can alo be oberved that DBCT alway outperfor DLT under erroneou uperviion. But with reference uperviion, DLT i ignificantly better than DBCT. Thi i becaue DBCT i learned on the training et and i not tuned to the reference during adaptation Training Stability Another notable obervation i that DBCT ay encounter intability in dicriinative update. In Fig., we plot the Table 2. WER% of DBCT and DLT with different uperviion. Superviion ML-SI MMI-SI Ref WER Bai-MLLR DBCT DLT MMI objective for 0 iteration of DBCT etiation and the correponding WER. With E=0.8, there i a draatic rie in the MMI objective after 5 iteration, while the perforance of DBCT on the teting et begin to drop. Thi intability can be relieved by etting the cont E to a larger value, which correpond to a aller tep ize in each iteration. Fig. alo how the MMI objective and WER with E=.2. In thi cae, DBCT etiation reain table within the 0 iteration. But we need ore iteration to reach the optial recognition reult. Moreover, we oberve that increaing the coefficient dienion P to 20 or 30 introduce ore intability into DBCT training. That i, the teting WER goe up quickly to 50% after everal iteration. That' why we et P to 0 in our experient. MMI Obj Obj E=0.8 Obj E=.2 WER E=0.8 WER E= iteration Fig.. Intability of DBCT etiation in ter of MMI objective and WER% in 0 iteration. 5. CONCLUSIONS AND FUTURE WORK In thi paper, we propoe an approach to etiating DBCT for eigenpace MLLR adaptation. The DBCT tranfor i learned on top of bai-mllr and applied in recognition to iprove the ML adaptation with additional dicriination. Experient how the effectivene of DBCT in iproving unupervied adaptation. In our future work, we will focu on the extenion of thi ethod to cluter adaptive training (CAT) [], where we can learn the dicriinative tranfor on the cluter cobination coefficient. 6. ACKNOWLEDGMENTS Thi work wa upported by the Intelligence Advanced Reearch Project Activity (IARPA) via Departent of Defene U.S. Ary Reearch Laboratory (DoD / ARL) contract nuber W9NF-2-C-005. The U.S. Governent i authorized to reproduce and ditribute reprint for Governental purpoe notwithtanding any copyright annotation thereon. Diclaier: The view and concluion contained herein are thoe of the author and hould not be interpreted a necearily repreenting the official policie or endoreent, either expreed or iplied, of IARPA, DoD/ARL, or the U.S. Governent WER%

5 7. REFERENCES [] M. Gale, Maxiu likelihood linear tranforation for HMM-baed peech recognition, Coputer Speech and Language, vol. 2, pp , 998. [2] K. T. Chen, W. W. Liau, H. M. Wang, and L. S. Lee, Fat peaker adaptation uing eigenpace-baed axiu likelihood linear regreion, in Proc. ICSLP, pp , [3] N. Wang, S. Lee, F. Seide, and L. S. Lee, Rapid peaker adaptation uing a priori knowledge by eigenpace analyi of MLLR paraeter, in Proc. ICASSP, pp , 200. [4] K. Viwewariah, V. Goel, and R. Gopinath, Maxiu Likelihood training of bae for rapid adaptation, unpublihed anucript, [5] L. Wang and P. C. Woodland, Dicriinative adaptive training uing the MPE criterion, in Proc. ASRU, pp , [6] L. Wang and P. C. Woodland, MPE-baed dicriinative linear tranfor for peaker adaptation, Coputer Speech and Language, vol. 22, pp , [7] K. Yu, M. Gale, and P. C. Woodland, Unupervied dicriinative adaptation uing dicriinative apping tranfor, in Proc. ICASSP, pp , [8] K. Yu, M. Gale, and P. C. Woodland, "Unupervied adaptation with dicriinative apping tranfor, IEEE Tranaction on Audio, Speech, and Language Proceing, vol. 7, no. 4, pp , [9] D. Povey, Dicriinative training for large vocabulary peech recognition, Ph.D. diertation, Cabridge Univ., Cabridge, U.K., [0] D. Povey and K. Yao, A bai repreentation of contrained MLLR tranfor for robut adaptation, Coputer Speech and Language, vol. 26, pp. 35-5, 202. [] M. Gale, "Cluter adaptive training of Hid Markov Model, IEEE Tranaction on Audio, Speech, and Language Proceing, vol. 8, no. 4, pp , 2000.