Vocoder-Free Non-parallel Conversion of Whispered Speech With Masked Cycle-Consistent Generative Adversarial Networks

  • Cycle-consistent generative adversarial networks have been widely used in non-parallel voice conversion (VC). Their ability to learn mappings between source and target features without relying on parallel training data eliminates the need for temporal alignments. However, most methods decouple the conversion of acoustic features from synthesizing the audio signal by using separate models for conversion and waveform synthesis. This work unifies conversion and synthesis into a single model, thereby eliminating the need for a separate vocoder. By leveraging cycle-consistent training and a self-supervised auxiliary training task, our model is able to efficiently generate converted high-quality raw audio waveforms. Subjective listening tests showed that our unified approach achieved improvements of up to 6.7% relative to the baseline in whispered VC. Mean opinion score predictions also yielded stable results in conventional VC (between 0.5% and 2.4% relative improvement).

Export metadata

Additional Services

Search Google Scholar
Metadaten
Author:Dominik Wagner, Ilja Baumann, Tobias BockletORCiD
DOI:https://doi.org/10.1007/978-3-032-02548-7_20
ISBN:978-3-032-02548-7
Publisher:Springer
Place of publication:Cham
Document Type:conference proceeding (article)
Language:English
Date of first Publication:2025/08/22
Reviewed:Begutachtet/Reviewed
Release Date:2025/11/04
Tag:cycle-consistency; generative adversarial networks; masking; voice conversion; whispered speech
Pagenumber:235-246
Konferenzangabe:Int’l Conference on Text, Speech and Dialogue (TSD)
institutes:Fakultät Informatik
Zentrum für Künstliche Intelligenz (KIZ)
Research Themes:Digitalisierung & Künstliche Intelligenz
Verstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.