Development of a Language Model for Medical Domain
- Language models are widely used as a representation of written language in various machine learning tasks, with the most commonly used model being Bidirectional Encoder Representations from Transformers (BERT). It was shown that the prediction quality strongly benefits from language model pre-training on domain-specific data. The publicly available models, though are always trained on Wikipedia, news or legal data, thereby missing the domain specific knowledge about medical terms. In this thesis, we will train a BERT language model on medical data and compare performance with domain-unspecific language models. The dataset used for this purpose is the Non-technical Summaries - International Statistical Classification of Diseases (NTS-ICD) task of classification of animal experiment descriptions into International Statistical Classification of Diseases (ICD) categories.
Author: | Manjil Shrestha |
---|---|
URN: | urn:nbn:de:hbz:1383-opus4-7405 |
Examiner: | Rolf Becker, Lukas Gilz |
Document Type: | Master Thesis |
Language: | English |
Date of Publication (online): | 2021/03/26 |
Date of first Publication: | 2021/03/26 |
Publishing Institution: | Hochschule Rhein-Waal |
Degree-granting institution: | Hochschule Rhein-Waal |
Date of final exam: | 2020/11/13 |
Release Date: | 2021/03/26 |
Tag: | Fine-tuning; German BERT NTS classification; ICD-10 Codes; Mult-label classification; Transfer learning |
Number of Pages: | 141 |
Faculties and Institutes: | Fakultät Kommunikation und Umwelt |
Dewey Decimal Classification: | 6 Technik, Medizin, angewandte Wissenschaften / 60 Technik |
0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 000 Informatik, Informationswissenschaft, allgemeine Werke | |
Licence (German): | ![]() |