Towards Self-Attention Understanding for Automatic Articulatory Processes Analysis in Cleft Lip and Palate Speech

  • Cleft lip and palate (CLP) speech presents unique challenges for automatic phoneme analysis due to its distinct acoustic characteristics and articulatory anomalies. We perform phoneme analysis in CLP speech using a pre-trained wav2vec 2.0 model with a multi-head self-attention classification module to capture long-range dependencies within the speech signal, thereby enabling better contextual understanding of phoneme sequences. We demonstrate the effectiveness of our approach in the classification of various articulatory processes in CLP speech. Furthermore, we investigate the interpretability of self-attention to gain insights into the model’s understanding of CLP speech characteristics. Our findings highlight the potential of the selfattention mechanisms for improving automatic phoneme analysis in CLP speech, paving the way for enhanced diagnostics, adding interpretability for therapists and affected patients.

Export metadata

Additional Services

Search Google Scholar
Metadaten
Author:Ilja BaumannORCiD, Dominik Wagner, Maria Schuster, Korbinian RiedhammerORCiD, Elmar Nöth, Tobias BockletORCiD
DOI:https://doi.org/10.21437/Interspeech.2024-2134
Document Type:conference proceeding (article)
Language:English
Date of first Publication:2024/06/30
Reviewed:Begutachtet/Reviewed
Release Date:2024/10/17
Tag:pathologic speech, cleft lip and palate, children’s speech, automatic assessment
Pagenumber:5
First Page:2430
Last Page:2434
Konferenzangabe:Interspeech 2024
institutes:Zentrum für Künstliche Intelligenz (KIZ)
Research Themes:Digitalisierung & Künstliche Intelligenz
Verstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.