Speaker Adaptation for End-To-End Speech Recognition Systems in Noisy Environments

  • We analyze the impact of speaker adaptation in end-to-end automatic speech recognition models based on transformers and wav2vec 2.0 under different noise conditions. By including speaker embeddings obtained from x-vector and ECAPA-TDNN systems, as well as i-vectors, we achieve relative word error rate improvements of up to 16.3% on LibriSpeech and up to 14.5% on Switchboard. We show that the proven method of concatenating speaker vectors to the acoustic features and supplying them as auxiliary model inputs remains a viable option to increase the robustness of end-to-end architectures. The effect on transformer models is stronger, when more noise is added to the input speech. The most substantial benefits for systems based on wav2vec 2.0 are achieved under moderate or no noise conditions. Both x-vectors and ECAPA-TDNN embeddings outperform i-vectors as speaker representations. The optimal embedding size depends on the dataset and also varies with the noise condition.

Export metadata

Additional Services

Search Google Scholar
Metadaten
Author:Dominik Wagner, Ilja Baumann, Sebastian P. BayerlORCiD, Korbinian Riedhammer, Tobias Bocklet
DOI:https://doi.org/10.48550/arXiv.2211.08774
ArXiv Id:http://arxiv.org/abs/2211.08774
Document Type:conference proceeding (article)
Language:English
Date of first Publication:2023/12/07
Reviewed:Begutachtet/Reviewed
Release Date:2024/07/08
Tag:speaker adaptation, automatic speech recognition, end-to-end systems, transformer, wav2vec 2.0
Pagenumber:6
Konferenzangabe:IEEE Workshop on Speech Recognition and Understanding (ASRU)
institutes:Fakultät Informatik
Licence (German):Keine Lizenz - Deutsches Urheberrecht gilt
Verstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.