Vocoder-Free Non-parallel Conversion of Whispered Speech With Masked Cycle-Consistent Generative Adversarial Networks
- Cycle-consistent generative adversarial networks have been widely used in non-parallel voice conversion (VC). Their ability to learn mappings between source and target features without relying on parallel training data eliminates the need for temporal alignments. However, most methods decouple the conversion of acoustic features from synthesizing the audio signal by using separate models for conversion and waveform synthesis. This work unifies conversion and synthesis into a single model, thereby eliminating the need for a separate vocoder. By leveraging cycle-consistent training and a self-supervised auxiliary training task, our model is able to efficiently generate converted high-quality raw audio waveforms. Subjective listening tests showed that our unified approach achieved improvements of up to 6.7% relative to the baseline in whispered VC. Mean opinion score predictions also yielded stable results in conventional VC (between 0.5% and 2.4% relative improvement).
| Author: | Dominik Wagner, Ilja Baumann, Tobias BockletORCiD |
|---|---|
| DOI: | https://doi.org/10.1007/978-3-032-02548-7_20 |
| ISBN: | 978-3-032-02548-7 |
| Publisher: | Springer |
| Place of publication: | Cham |
| Document Type: | conference proceeding (article) |
| Language: | English |
| Date of first Publication: | 2025/08/22 |
| Reviewed: | Begutachtet/Reviewed |
| Release Date: | 2025/11/04 |
| Tag: | cycle-consistency; generative adversarial networks; masking; voice conversion; whispered speech |
| Pagenumber: | 235-246 |
| Konferenzangabe: | Int’l Conference on Text, Speech and Dialogue (TSD) |
| institutes: | Fakultät Informatik |
| Zentrum für Künstliche Intelligenz (KIZ) | |
| Research Themes: | Digitalisierung & Künstliche Intelligenz |
