Filtern
Dokumenttyp
Sprache
- Englisch (3)
Volltext vorhanden
- nein (3)
Gehört zur Bibliographie
- ja (3)
Schlagworte
- Desinformation (3)
- Fehlinformation (3)
- COVID-19 (1)
- Information Retrieval (1)
We describe the outcome of the 2023 edition of the CheckThat!Lab at CLEF. We focus on subjectivity (Task 2), which has been proposed for the first time. It aims at fostering the technology for the identification of subjective text fragments in news articles. For that, we produced corpora consisting of 9,530 manually-annotated sentences, covering six languages - Arabic, Dutch, English, German, Italian, and Turkish. Task 2 attracted 12 teams, which submitted a total of 40 final runs covering all languages. The most successful approaches addressed the task using state-of-the-art multilingual transformer models, which were fine-tuned on language-specific data. Teams also experimented with a rich set of other neural architectures, including foundation models, zero-shot classifiers, and standard transformers, mainly coupled with data augmentation and multilingual training strategies to address class imbalance. We publicly release all the datasets and evaluation scripts, with the purpose of promoting further research on this topic.
The CLEF-2025 CheckThat! Lab
(2025)
The CheckThat! lab aims to advance the development of innovative technologies designed to identify and to counteract online disinformation and manipulation efforts across various languages and platforms. The first five editions of the CheckThat! lab focused on the main tasks of the information verification pipeline: check-worthiness, evidence retrieval and pairing, and verification. Since the 2023 edition, the lab has broadened the focus and addressed new problems on auxiliary tasks supporting research and decision-making during the verification process. In the 2025 edition of the lab, we consider tasks at the core of the verification pipeline again as well as auxiliary tasks: Task 1 is on identification of subjectivity (a follow up of the CheckThat! 2024 edition), Task 2 is on claim normalization, Task 3 addresses fact-checking numerical claims, and Task 4 focuses on scientific web discourse processing. These tasks represent challenging classification and retrieval problems at the document and at the span level, including multilingual settings.
This paper presents the eighth edition of the CheckThat! lab, part of the 2025 Conference and Labs of the Evaluation Forum (CLEF). As in previous editions of CheckThat!, the lab offers tasks from the core of the verification pipeline, including check-worthiness, identifying previously fact-checked claims, supporting evidence retrieval, and claim verification as well as auxiliary tasks addressing different facets of individual steps of the pipeline: Task 1 is on identification of subjectivity (a follow-up of the CheckThat! 2024 edition), which is related to the check-worthiness task, Task 2 is on claim normalization, Task 3 addresses fact-checking numerical claims, and Task 4 focuses on scientific web discourse processing. These challenging classification and retrieval problems are offered in different mono-, multi- and crosslingual settings covering more than 20 languages. This year, CheckThat! was one of the most popular labs at CLEF-2025 in terms of team registrations: 177 teams registered, almost half of them actually participating (a total of 83 teams) and 54 submitted system description papers.