TY - CHAP A1 - Nakov, Preslav A1 - Da San Martino, Giovanni A1 - Elsayed, Tamer A1 - Barrón-Cedeño, Alberto A1 - Míguez, Rubén A1 - Shaar, Shaden A1 - Alam, Firoj A1 - Haouari, Fatima A1 - Hasanain, Maram A1 - Mansour, Watheq A1 - Hamdan, Bayan A1 - Sheikh Ali, Zien A1 - Babulkov, Nikolay A1 - Nikolov, Alex A1 - Koshore Shahi, Gautam A1 - Struß, Julia Maria A1 - Mandl, Thomas A1 - Kutlu, Mucahid A1 - Selim Kartal, Yavuz T1 - Overview of the CLEF–2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News T2 - Experimental IR Meets Multilinguality, Multimodality, and Interaction N2 - We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, English, Spanish, and Turkish. Task 1 asks to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics (in all five languages). Task 2 asks to determine whether a claim in a tweet can be verified using a set of previously fact-checked claims (in Arabic and English). Task 3 asks to predict the veracity of a news article and its topical domain (in English). The evaluation is based on mean average precision or precision at rank k for the ranking tasks, and macro-F1 for the classification tasks. This was the most popular CLEF-2021 lab in terms of team registrations: 132 teams. Nearly one-third of them participated: 15, 5, and 25 teams submitted official runs for tasks 1, 2, and 3, respectively. KW - Fact-checking KW - Check-worthiness estimation KW - Desinformation KW - Fehlinformation Y1 - 2021 SN - 978-3-030-85251-1 U6 - https://doi.org/10.1007/978-3-030-85251-1_19 SP - 264 EP - 291 PB - Springer CY - Cham ER - TY - CHAP A1 - Nakov, Preslav A1 - Da San Martino, Giovanni A1 - Elsayed, Tamer A1 - Barrón-Cedeño, Alberto A1 - Míguez, Rúben A1 - Shaar, Shaden A1 - Alam, Firoj A1 - Haouari, Fatima A1 - Hasanain, Maram A1 - Babulkov, Nikolay A1 - Nikolov, Alex A1 - Shahi, Gautam Kishore A1 - Struß, Julia Maria A1 - Mandl, Thomas T1 - The CLEF-2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News T2 - Advances in Information Retrieval N2 - We describe the fourth edition of the CheckThat! Lab, part of the 2021 Cross-Language Evaluation Forum (CLEF). The lab evaluates technology supporting various tasks related to factuality, and it is offered in Arabic, Bulgarian, English, and Spanish. Task 1 asks to predict which tweets in a Twitter stream are worth fact-checking (focusing on COVID-19). Task 2 asks to determine whether a claim in a tweet can be verified using a set of previously fact-checked claims. Task 3 asks to predict the veracity of a target news article and its topical domain. The evaluation is carried out using mean average precision or precision at rank k for the ranking tasks, and F1 for the classification tasks. KW - Fake news KW - Fact-checking KW - Falschmeldung KW - Desinformation KW - Fehlinformation Y1 - 2021 SN - 978-3-030-72240-1 U6 - https://doi.org/10.1007/978-3-030-72240-1_75 SP - 639 EP - 649 PB - Springer CY - Cham ER - TY - CHAP A1 - Nakov, Preslav A1 - Barrón-Cedño, Alberto A1 - Da San Martino, Giovanni A1 - Alam, Firoj A1 - Struß, Julia Maria A1 - Mandl, Thomas A1 - Míguez, Rubén A1 - Caselli, Tommaso A1 - Kutlu, Mucahid A1 - Zaghouani, Wajdi A1 - Li, Chengkai A1 - Shaar, Shaden A1 - Shahi, Gautam Kishore A1 - Mubarak, Hamdy A1 - Nikolov, Alex A1 - Babulkov, Nikolay A1 - Kartal, Yavuz Selim A1 - Beltrán, Javier T1 - The CLEF-2022 CheckThat! Lab on Fighting the COVID-19 Infodemic and Fake News Detection T2 - Advances in Information Retrieval. ECIR 2022 N2 - The fifth edition of the CheckThat! Lab is held as part of the 2022 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting various factuality tasks in seven languages: Arabic, Bulgarian, Dutch, English, German, Spanish, and Turkish. Task 1 focuses on disinformation related to the ongoing COVID-19 infodemic and politics, and asks to predict whether a tweet is worth fact-checking, contains a verifiable factual claim, is harmful to the society, or is of interest to policy makers and why. Task 2 asks to retrieve claims that have been previously fact-checked and that could be useful to verify the claim in a tweet. Task 3 is to predict the veracity of a news article. Tasks 1 and 3 are classification problems, while Task 2 is a ranking one. KW - Desinformation KW - Fehlinformation KW - COVID-19 KW - Fact-checking KW - Check-worthiness estimation Y1 - 2022 SN - 978-3-030-99739-7 U6 - https://doi.org/10.1007/978-3-030-99739-7_52 SN - 0302-9743 SP - 416 EP - 428 PB - Springer CY - Cham ER - TY - CHAP A1 - Nakov, Preslav A1 - Barrón-Cedño, Alberto A1 - Da San Martino, Giovanni A1 - Alam, Firoj A1 - Struß, Julia Maria A1 - Mandl, Thomas A1 - Míguez, Rubén A1 - Caselli, Tommaso A1 - Kutlu, Mucahid A1 - Zaghouani, Wajdi A1 - Li, Chengkai A1 - Shaar, Shaden A1 - Shahi, Gautam Kishore A1 - Mubarak, Hamdy A1 - Nikolov, Alex A1 - Babulkov, Nikolay A1 - Kartal, Yavuz Selim A1 - Wiegand, Michael A1 - Siegel, Melanie A1 - Köhler, Juliane T1 - Overview of the CLEF-2022 CheckThat! Lab on Fighting the COVID-19 Infodemic and Fake News Detection T2 - Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2022. N2 - We describe the fifth edition of the CheckThat! lab, part of the 2022 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality in multiple languages: Arabic, Bulgarian, Dutch, English, German, Spanish, and Turkish. Task 1 asks to identify relevant claims in tweets in terms of check-worthiness, verifiability, harmfullness, and attention-worthiness. Task 2 asks to detect previously fact-checked claims that could be relevant to fact-check a new claim. It targets both tweets and political debates/speeches. Task 3 asks to predict the veracity of the main claim in a news article. CheckThat! was the most popular lab at CLEF-2022 in terms of team registrations: 137 teams. More than one-third (37%) of them actually participated: 18, 7, and 26 teams submitted 210, 37, and 126 official runs for tasks 1, 2, and 3, respectively. KW - Check-worthiness estimation KW - Fact-checking KW - Desinformation KW - Fehlinformation KW - COVID-19 Y1 - 2022 SN - 978-3-031-13643-6 U6 - https://doi.org/10.1007/978-3-031-13643-6_29 SN - 0302-9743 SP - 495 EP - 520 PB - Springer CY - Cham ER - TY - CHAP A1 - Barrón-Cedeño, Alberto A1 - Alam, Firoj A1 - Caselli, Tommaso A1 - Da San Martino, Giovanni A1 - Elsayed, Tamer A1 - Galassi, Andrea A1 - Haouari, Fatima A1 - Ruggeri, Federico A1 - Struß, Julia Maria A1 - Nath Nandi, Rabindra A1 - Cheema, Gullal S. A1 - Azizov, Dilshod A1 - Nakov, Preslav T1 - The CLEF-2023 CheckThat! Lab: Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority T2 - Advances in Information Retrieval : 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland, April 2–6, 2023, Proceedings, Part III N2 - The five editions of the CheckThat! lab so far have focused on the main tasks of the information verification pipeline: check-worthiness, evidence retrieval and pairing, and verification. The 2023 edition of the lab zooms into some of the problems and - for the first time - it offers five tasks in seven languages (Arabic, Dutch, English, German, Italian, Spanish, and Turkish): Task 1 asks to determine whether an item, text or a text plus an image, is check-worthy; Task 2 requires to assess whether a text snippet is subjective or not; Task 3 looks for estimating the political bias of a document or a news outlet; Task 4 requires to determine the level of factuality of a document or a news outlet; and Task 5 is about identifying authorities that should be trusted to verify a contended claim. KW - Desinformation KW - Fehlinformation KW - Bias Y1 - 2023 SN - 978-3-031-28241-6 U6 - https://doi.org/10.1007/978-3-031-28241-6_59 SN - 0302-9743 SP - 509 EP - 517 PB - Springer CY - Cham ER -