The search result changed since you submitted your search request. Documents might be displayed in a different sort order.
  • search hit 1 of 36
Back to Result List

Development of a Language Model for Medical Domain

  • Language models are widely used as a representation of written language in various machine learning tasks, with the most commonly used model being Bidirectional Encoder Representations from Transformers (BERT). It was shown that the prediction quality strongly benefits from language model pre-training on domain-specific data. The publicly available models, though are always trained on Wikipedia, news or legal data, thereby missing the domain specific knowledge about medical terms. In this thesis, we will train a BERT language model on medical data and compare performance with domain-unspecific language models. The dataset used for this purpose is the Non-technical Summaries - International Statistical Classification of Diseases (NTS-ICD) task of classification of animal experiment descriptions into International Statistical Classification of Diseases (ICD) categories.

Download full text files

  • This is a MSc. Thesis report. The topic is "Development of a Language Model for medical domain". Here, we fine-tune the existing German BERT language model with several medical articles related to diagnosis, symptoms, diseases which are collected from the web.

Export metadata

Additional Services

Share in Twitter Search Google Scholar
Author:Manjil Shrestha
Examiner:Rolf Becker, Lukas Gilz
Document Type:Master Thesis
Date of Publication (online):2021/03/26
Date of first Publication:2021/03/26
Publishing Institution:Hochschule Rhein-Waal
Degree-granting institution:Hochschule Rhein-Waal
Date of final exam:2020/11/13
Release Date:2021/03/26
Tag:Fine-tuning; German BERT NTS classification; ICD-10 Codes; Mult-label classification; Transfer learning
Number of Pages:141
Faculties and Institutes:Fakult├Ąt Kommunikation und Umwelt
Dewey Decimal Classification:6 Technik, Medizin, angewandte Wissenschaften / 60 Technik
0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 000 Informatik, Informationswissenschaft, allgemeine Werke
Licence (German):License LogoCC BY-SA 4.0 International - Namensnennung-Weitergabe unter gleichen Bedingungen