Cross lingual transfer learning does not improve aphasic speech recognition

  • In addressing the particular linguistic challenges posed by patients suffering from aphasia, a language disorder, this paper proposes a fine-tuning approach to enhance the speech recognition capabilities of existing models. The available aphasic research data in German is highly limited. To address this constraint, we propose a cross-lingual transfer approach to utilize English data to improveIn addressing the particular linguistic challenges posed by patients suffering from aphasia, a language disorder, this paper proposes a fine-tuning approach to enhance the speech recognition capabilities of existing models. The available aphasic research data in German is highly limited. To address this constraint, we propose a cross-lingual transfer approach to utilize English data to improve performance in German. This advancement aims to support the development of a therapy platform tailored for patients with aphasia. For the base speech recognition model, we choose to use OpenAI’s Whisper model, and for fine-tuning, we make use of TalkBank’s AphasiaBank. The experimental findings demonstrate that the transcription of aphasic audio with Whisper is less successful than non-aphasic audio. However, fine-tuning the transcription in the respective language resulted in an enhancement of its quality. In contrast, fine-tuning the transcription in another language and expecting a transfer of the learned aphasic speech properties led to a deterioration in its quality.show moreshow less

Download full text files

Export metadata

Additional Services

Share in Twitter Search Google Scholar Statistics
Metadaten
Author:Sara Mühlhausen, Sarah Gomez, Norina LauerORCiDGND, Timo BaumannORCiDGND
URN:urn:nbn:de:bvb:898-opus4-80518
URL / DOI:https://www.essv.de/pdf/2025_77_84.pdf
DOI:https://doi.org/10.35096/othr/pub-8051
ISBN:978-3-95908-803-9
ISSN:0940-6832
Parent Title (English):Elektronische Sprachsignalverarbeitung 2025: Tagungsband der 36. Konferenz Halle/Saale, 05.–07. MÄRZ 2025
Publisher:TUDpress
Place of publication:Dresden
Editor:Sven Grawunder
Document Type:conference proceeding (article)
Language:English
Year of first Publication:2025
Publishing Institution:Ostbayerische Technische Hochschule Regensburg
Release Date:2025/04/28
GND Keyword:Automatische Spracherkennung; Sprachdialogsystem; Aphasie
Pagenumber:8
Andere Schriftenreihe:Studientexte zur Sprachkommunikation, 110
Institutes:Fakultät Informatik und Mathematik
Fakultät Sozial- und Gesundheitswissenschaften
research focus:Information und Kommunikation
Licence (German):Keine Lizenz - Es gilt das deutsche Urheberrecht: § 53 UrhG
Einverstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.