Refine
Has Fulltext
- yes (1)
Year of publication
- 2021 (1)
Document Type
- Master's Thesis (1)
Language
- English (1)
Is part of the Bibliography
- yes (1)
Keywords
- Transformer (1) (remove)
Institute
This thesis focuses on the automatic classification of research methods used in scientific articles in the domain of Information Systems, and studies the effectiveness of a state-of-the-art long-document Transformer technique, Longformer, on the multi-label classification task.
In recent years, the task of automatically extracting knowledge from academic articles has become more and more popular. However, as far as the author knows, due to the limitations of the max input sequence length of Transformer models, there has not yet been a comprehensive study on the classification of research methods using Transformer models, which have made extraordinary achievements in various Natural Language Processing (NLP) fields. Therefore, this thesis establishes an artifact that uses a modified Transformer model, Lomgformer, which can proceed with long-sequence inputs, to discuss its effectiveness and possible limitations. Additionally, this thesis also discusses and evaluates the performance of other benchmark models, such as the traditional Transformer - BERT and RoBERTa, and a non-transfer-learning model based on the architecture of convolutional neural network (CNN), which has been proven to have good performance by previous researchers.
As a result, this thesis proves that the Longformer-based artifact can effectively improve the classification performance for the scientific articles, and surpasses all the other models, not only the traditional Transformer models but also the models presented in the literature from the previous researchers.