Cross lingual transfer learning does not improve aphasic speech recognition
- In addressing the particular linguistic challenges posed by patients suffering from aphasia, a language disorder, this paper proposes a fine-tuning approach to enhance the speech recognition capabilities of existing models. The available aphasic research data in German is highly limited. To address this constraint, we propose a cross-lingual transfer approach to utilize English data to improveIn addressing the particular linguistic challenges posed by patients suffering from aphasia, a language disorder, this paper proposes a fine-tuning approach to enhance the speech recognition capabilities of existing models. The available aphasic research data in German is highly limited. To address this constraint, we propose a cross-lingual transfer approach to utilize English data to improve performance in German. This advancement aims to support the development of a therapy platform tailored for patients with aphasia. For the base speech recognition model, we choose to use OpenAI’s Whisper model, and for fine-tuning, we make use of TalkBank’s AphasiaBank. The experimental findings demonstrate that the transcription of aphasic audio with Whisper is less successful than non-aphasic audio. However, fine-tuning the transcription in the respective language resulted in an enhancement of its quality. In contrast, fine-tuning the transcription in another language and expecting a transfer of the learned aphasic speech properties led to a deterioration in its quality.…


| Author: | Sara Mühlhausen, Sarah Gomez, Norina LauerORCiDGND, Timo BaumannORCiDGND |
|---|---|
| URN: | urn:nbn:de:bvb:898-opus4-80518 |
| URL / DOI: | https://www.essv.de/pdf/2025_77_84.pdf |
| DOI: | https://doi.org/10.35096/othr/pub-8051 |
| ISBN: | 978-3-95908-803-9 |
| ISSN: | 0940-6832 |
| Parent Title (English): | Elektronische Sprachsignalverarbeitung 2025: Tagungsband der 36. Konferenz Halle/Saale, 05.–07. MÄRZ 2025 |
| Publisher: | TUDpress |
| Place of publication: | Dresden |
| Editor: | Sven Grawunder |
| Document Type: | conference proceeding (article) |
| Language: | English |
| Year of first Publication: | 2025 |
| Publishing Institution: | Ostbayerische Technische Hochschule Regensburg |
| Release Date: | 2025/04/28 |
| GND Keyword: | Automatische Spracherkennung; Sprachdialogsystem; Aphasie |
| Pagenumber: | 8 |
| Andere Schriftenreihe: | Studientexte zur Sprachkommunikation, 110 |
| Institutes: | Fakultät Informatik und Mathematik |
| Fakultät Sozial- und Gesundheitswissenschaften | |
| research focus: | Information und Kommunikation |
| Licence (German): |


