Generative Models for Improved Naturalness, Intelligibility, and Voicing of Whispered Speech

  • This work adapts two recent architectures of generative models and evaluates their effectiveness for the conversion of whispered speech to normal speech. We incorporate the normal target speech into the training criterion of vector-quantized variational autoencoders (VQ-VAEs) and Mel-GANs, thereby conditioning the systems to recover voiced speech from whispered inputs. Objective and subjective quality measures indicate that both VQ-VAEs and MelGANs can be modified to perform the conversion task. We find that the proposed approaches significantly improve the Mel cepstral distortion (MCD) metric by at least 25% relative to a Disco-GAN baseline. Subjective listening tests suggest that the MelGAN-based system significantly improves naturalness, intelligibility, and voicing compared to the whispered input speech. A novel evaluation measure based on differences between latent speech representations also indicates that our MelGAN-based approach yields improvements relative to the baseline.

Export metadata

Additional Services

Search Google Scholar
Metadaten
Author:Dominik Wagner, Sebastian P. BayerlORCiD, Hector A. Cordourier Maruri, Tobias BockletORCiD
DOI:https://doi.org/10.1109/SLT54892.2023.10022796
ISBN:979-8-3503-9690-4
ArXiv Id:http://arxiv.org/abs/2212.01775v2
Parent Title (English):2022 IEEE Spoken Language Technology Workshop (SLT)
Publisher:IEEE
Document Type:conference proceeding (article)
Language:English
Reviewed:Begutachtet/Reviewed
Release Date:2024/08/29
Tag:GAN; VAE; generative models; speech conversion; whispered speech
Pagenumber:943 - 948
First Page:943
Last Page:948
Konferenzangabe:2022 IEEE Spoken Language Technology Workshop (SLT), 09-12 January 2023, Doha, Qatar
institutes:Fakultät Informatik
Zentrum für Künstliche Intelligenz (KIZ)
Research Themes:Digitalisierung & Künstliche Intelligenz
Verstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.