Detecting Dysfluencies in Stuttering Therapy Using wav2vec 2.0
- Stuttering is a varied speech disorder that harms an individual’s
communication ability. Persons who stutter (PWS) often use
speech therapy to cope with their condition. Improving speech
recognition systems for people with such non-typical speech or
tracking the effectiveness of speech therapy would require systems that can detect dysfluencies while at the same time being
able to detect speech techniques acquired in therapy.
This paper shows that fine-tuning wav2vec 2.0 [1] for
the classification of stuttering on a sizeable English corpus
containing stuttered speech, in conjunction with multi-task
learning, boosts the effectiveness of the general-purpose
wav2vec 2.0 features for detecting stuttering in speech; both
within and across languages. We evaluate our method on
FluencyBank , [2] and the German therapy-centric Kassel
State of Fluency (KSoF) [3] dataset by training Support Vector
Machine classifiers using features extracted from the finetuned models for six different stuttering-related event types:
blocks,Stuttering is a varied speech disorder that harms an individual’s
communication ability. Persons who stutter (PWS) often use
speech therapy to cope with their condition. Improving speech
recognition systems for people with such non-typical speech or
tracking the effectiveness of speech therapy would require systems that can detect dysfluencies while at the same time being
able to detect speech techniques acquired in therapy.
This paper shows that fine-tuning wav2vec 2.0 [1] for
the classification of stuttering on a sizeable English corpus
containing stuttered speech, in conjunction with multi-task
learning, boosts the effectiveness of the general-purpose
wav2vec 2.0 features for detecting stuttering in speech; both
within and across languages. We evaluate our method on
FluencyBank , [2] and the German therapy-centric Kassel
State of Fluency (KSoF) [3] dataset by training Support Vector
Machine classifiers using features extracted from the finetuned models for six different stuttering-related event types:
blocks, prolongations, sound repetitions, word repetitions,
interjections, and – specific to therapy – speech modifications.
Using embeddings from the fine-tuned models leads to relative classification performance gains up to 27% w.r.t. F1-score.…


| Author: | Sebastian Peter BayerlORCiD, Dominik Wagner, Elmar Noeth, Korbinian RiedhammerORCiD |
|---|---|
| DOI: | https://doi.org/10.21437/Interspeech.2022-10908 |
| Parent Title (English): | Interspeech 2022 |
| Publisher: | ISCA |
| Place of publication: | ISCA |
| Document Type: | conference proceeding (article) |
| Language: | English |
| Reviewed: | Begutachtet/Reviewed |
| Release Date: | 2024/08/02 |
| Tag: | computational paralinguistics; pathological speech; stuttering |
| Pagenumber: | 5 |
| First Page: | 2868 |
| Last Page: | 2872 |
| Konferenzangabe: | Interspeech 2022, 18-22 September 2022, Incheon, Korea |
| institutes: | Fakultät Informatik |
| Zentrum für Künstliche Intelligenz (KIZ) | |
| Research Themes: | Digitalisierung & Künstliche Intelligenz |
