TY - JOUR A1 - Simbeck, Katharina T1 - They shall be fair, transparent, and robust: auditing learning analytics systems JF - AI and Ethics N2 - In the near future, systems, that use Artificial Intelligence (AI) methods, such as machine learning, are required to be certified or audited for fairness if used in ethically sensitive fields such as education. One example of those upcoming regulatory initiatives is the European Artificial Intelligence Act. Interconnected with fairness are the notions of system transparency (i.e. how understandable is the system) and system robustness (i.e. will similar inputs lead to similar results). Ensuring fairness, transparency, and robustness requires looking at data, models, system processes, and the use of systems as the ethical implications arise at the intersection between those. The potential societal consequences are domain specific, it is, therefore, necessary to discuss specifically for Learning Analytics (LA) what fairness, transparency, and robustness mean and how they can be certified. Approaches to certifying and auditing fairness in LA include assessing datasets, machine learning models, and the end-to-end LA process for fairness, transparency, and robustness. Based on Slade and Prinsloo’s six principals for ethical LA, relevant audit approaches will be deduced. Auditing AI applications in LA is a complex process that requires technical capabilities and needs to consider the perspectives of all stakeholders. This paper proposes a comprehensive framework for auditing AI applications in LA systems from the perspective of learners' autonomy, provides insights into different auditing methodologies, and emphasizes the importance of reflection and dialogue among providers, buyers, and users of these systems to ensure their ethical and responsible use. KW - Fairness KW - Fairness KW - Learning analytics KW - Fairness audit KW - AI audit KW - Information and Computing Sciences Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:523-19076 SN - 2730-5953 SN - 2730-5961 VL - 4 IS - 2 SP - 555 EP - 571 PB - Springer Nature ER - TY - RPRT A1 - Simbeck, Katharina A1 - Wehner, Marius A1 - Fernsel, Linda A1 - Schmodde, Lynn A1 - Kalff, Yannick A1 - Georgy, Jasmin A1 - Resner, Monika T1 - Leitfaden zur Überprüfung von KI im Bildungsbereich auf Fairness N2 - Der Auditierungsprozess von KI-Systemen im Bildungsbereich umfasst die Abgrenzung der Systembestandteile, die Identifizierung möglicher Risiken, die Definition von Auditierungskriterien, die Vorbereitung der Überprüfung sowie die Überprüfung der Auditierungskriterien und ein kontinuierliches Monitoring. Die Auditierungskriterien sind domänenspezifisch und angelehnt an die sechs Prinzipien für ethische Learning Analytics Anwendungen nach Slade und Prinsloo (2013). Zur Überprüfung der Kriterien können unter anderem die Dokumentation des KI-Systems gesichtet, der Quellcode analysiert, die dem Algorithmus zugrundeliegenden Daten untersucht und datenbasierte Tests durchgeführt werden. Die Ziele des Audits und der erwünschte Nutzen sollten an die Lernenden kommuniziert werden. Nicht alle Lernenden haben den gleichen Wissensstand in Bezug auf KI-Systeme und können Bedenken gegenüber dem Einsatz von diesen haben (Ifenthaler & Schumacher, 2019). Die Prüfung des Systems durch eine externe Institution kann einen positiven Einfluss auf die Wahrnehmung der Lernenden haben. Lernende empfinden ein System als fairer, wenn sie nachvollziehen können, dass das KI-System geprüft wurde. Lernende nehmen KI-Systeme umso positiver wahr, je mehr Informationen sie erhalten – auch wenn sie auf mögliche Nachteile hingewiesen werden. Die transparente Offenlegung möglicher Nachteile (z. B. technische Grenzen) beim Einsatz eines KI-Systems in der Bildung ist aus ethischer Sicht unabdingbar. Durch regelmäßige Audits wird sichergestellt, dass die KI-Systeme in technischer Hinsicht funktionieren und gleichzeitig fair sind. Zugleich wird die Akzeptanz durch die Studierenden verbessert. Universitäten profitieren daher von Audits ihrer KI-Systeme. N2 - The audit process of AI systems in the education sector involves delineating the system components, identifying potential risks, defining audit criteria, preparing for the assessment, conducting the audit criteria review, and continuous monitoring. The audit criteria are domain-specific and aligned with the six principles for ethical learning analytics applications outlined by Slade and Prinsloo (2013). Activities such as reviewing documentation of the AI system, analyzing source code, examining the algorithm's underlying data, and conducting data-based tests can be employed to verify the criteria. The objectives of the audit and the desired benefits should be communicated to the learners. Not all learners have the same level of knowledge regarding AI systems and may have concerns about their use (Ifenthaler & Schumacher, 2019). Examining the system by an external institution can positively influence learners' perceptions. Learners perceive a system as fairer when they understand that the AI system has been audited. Learners view AI systems more positively when they receive more information—even when potential drawbacks are pointed out. Transparent disclosure of potential disadvantages (e.g., technical limitations) in using an AI system in education is essential from an ethical standpoint. Regular audits ensure that AI systems function technically while also being fair. Simultaneously, acceptance by students is enhanced. Therefore, universities benefit from audits of their AI systems. KW - Leitfaden KW - Auditierung KW - Fairness KW - Learning Analytics KW - Bildung KW - Einführung KW - Fairness KW - Bildung KW - Künstliche Intelligenz Y1 - U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:523-18115 ER - TY - JOUR A1 - Köchling, Alina A1 - Riazy, Shirin A1 - Wehner, Marius Claus A1 - Simbeck, Katharina T1 - Highly Accurate, But Still Discriminatory JF - Business & Information Systems Engineering N2 - The study aims to identify whether algorithmic decision making leads to unfair (i.e., unequal) treatment of certain protected groups in the recruitment context. Firms increasingly implement algorithmic decision making to save costs and increase efficiency. Moreover, algorithmic decision making is considered to be fairer than human decisions due to social prejudices. Recent publications, however, imply that the fairness of algorithmic decision making is not necessarily given. Therefore, to investigate this further, highly accurate algorithms were used to analyze a pre-existing data set of 10,000 video clips of individuals in self-presentation settings. The analysis shows that the under-representation concerning gender and ethnicity in the training data set leads to an unpredictable overestimation and/or underestimation of the likelihood of inviting representatives of these groups to a job interview. Furthermore, algorithms replicate the existing inequalities in the data set. Firms have to be careful when implementing algorithmic video analysis during recruitment as biases occur if the underlying training data set is unbalanced. KW - Künstliche Intelligenz KW - Fairness KW - Bias KW - Artificial algorithm decision making KW - Recruitment KW - Asynchronous video interview KW - Ethics KW - HR analytics KW - Artificial intelligence KW - Bias KW - Ethik KW - Personalbeschaffung KW - Personalorganisation Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:523-17115 SN - 2363-7005 SN - 1867-0202 VL - 63 IS - 1 SP - 39 EP - 54 PB - Springer Fachmedien Wiesbaden ER - TY - JOUR A1 - Simbeck, Katharina T1 - Publisher Correction: They shall be fair, transparent, and robust: auditing learning analytics systems JF - AI and Ethics Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:523-19855 SN - 2730-5953 SN - 2730-5961 VL - 4 IS - 2 SP - 573 EP - 573 PB - Springer Nature CY - Cham ER - TY - CHAP A1 - Fernsel, Linda A1 - Kalff, Yannick A1 - Simbeck, Katharina T1 - Audits for Trust: An Auditability Framework for AI-Based Learning Analytics Systems T2 - Proceedings of the 17th International Conference on Computer Supported Education - Volume 2: CSEDU N2 - Audits contribute to the trustworthiness of Learning Analytics (LA) systems that integrate Artificial Intelligence (AI) and may be legally required in the future. We argue that the efficacy of an audit depends on the auditability of the audited system. Therefore, systems need to be designed with auditability in mind. We present a framework for assessing the auditability of AI-integrating systems in education that consists of three parts: (1) verifiable claims about the validity, utility and ethics of the system, (2) evidence on subjects (data, models, or the system) in different types (documentation, raw sources and logs) to back or refute claims, (3) means to validate evidence such as technical APIs, monitoring tools, or explainable AI principles must be accessible to auditors. We apply the framework to assess the auditability of the Learning Management System Moodle, which supports an AIintegrating dropout prediction system. Moodle’s auditability is limited by incomplete documentation, insufficient monitoring capabilities, and a lack of available test data. KW - Audit KW - Auditability KW - Artificial Intelligence KW - Learning Analytics Y1 - 2025 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:523-20790 VL - 2 SP - 51 EP - 62 PB - SciTePress ER - TY - CHAP A1 - Fernsel, Linda A1 - Kalff, Yannick A1 - Simbeck, Katharina T1 - Where Is the Evidence? A Plugin for Auditing Moodle’s Learning Analytics T2 - Proceedings of the 16th International Conference on Computer Supported Education (CSEDU 2024) N2 - The paper presents the work-in-progress development of a Moodle plugin to improve the auditability of Moodle’s Learning Analytics component. Future legislation, such as the EU AI Act, will require audits and “conformity assessments” of “high-risk” AI systems. Educational applications can be considered high-risk systems due to their important role in individual life and career paths. Therefore, their correctness, fairness, and efficiency must be assessed. However, auditing of the Learning Analytics functions in Moodle is limited. No suitable test-data is available, models and configurations are not persistent and only aggregated quality metrics are returned that are insufficient to assess fairness. The plugin addresses these issues and provides a data interface to extract data for audits. The plugin allows to a) upload and select data for the audit, b) clearly differentiate between model configuration and trained models, c) keep trained models, their configuration and underlying data for future inspections and comparisons, and finally, d) the plugin saves raw predictions for further analysis. The plugin enables the audit of Moodle’s Learning Analytics and its underlying AI models and contributes to increased fairness and trustworthiness of Learning Analytics as well as its legally compliant application. KW - Auditability KW - Artificial Intelligence KW - Learning Analytics KW - Moodle KW - Plugin Development Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:523-20818 VL - 2 SP - 262 EP - 269 PB - SciTePress ER - TY - INPR A1 - Simbeck, Katharina A1 - Schauer, Sophie A1 - Fernsel, Linda T1 - The System Admin's Perspective: A Discussion on AI in Education with LMS Admins N2 - Many higher education institutions (HEIs) are implementing or reviewing the implementation of predictive Learning Analytics to improve learning processes, identify students at risk or provide personalized learning paths. The responsibility for selecting, reviewing, implementing and supporting those systems falls on system administrators, an under-researched stakeholder group of Artificial Intelligence in higher education. In this paper, we summarize qualitative insights from a workshop with system administrators in German HEIs. We find that the system administrators are highly aware not only of system requirements but also of the needs of various shareholder groups such as institutional leadership, learners and educators and that they put high emphasis on ethical, transparent and compliant system use. We conclude that system administrators should be involved more in research on the use of technology in education and that AI systems used in education need to provide possibilities to sufficiently test the system, including anonymous yet realistic test scenarios and data. KW - Predictive Learning Analytics KW - Ethical Learning Analytics KW - Learning Management Systems KW - Moodle KW - AI audit KW - Moodle Y1 - U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:523-20139 UR - https://ieeexplore.ieee.org/document/10837664 N1 - Preprint version of this published article: https://ieeexplore.ieee.org/abstract/document/10837664/ ER - TY - CHAP A1 - Kalff, Yannick A1 - Simbeck, Katharina ED - Kaya, Mesut ED - Bogers, Toine ED - Bied, Guillaume ED - Johnson, Chris ED - Decorte, Jens-Joris T1 - Explained, yet misunderstood: How AI Literacy shapes HR Managers' interpretation of User Interfaces in Recruiting Recommender Systems T2 - Proceedings of the 5th Workshop on Recommender Systems for Human Resources (RecSys in HR 2025) N2 - AI-based recommender systems increasingly influence recruitment decisions. Thus, transparency and responsible adoption in Human Resource Management (HRM) are critical. This study examines how HR managers' AI literacy influences their subjective perception and objective understanding of explainable AI (XAI) elements in recruiting recommender dashboards. In an online experiment, 410 German-based HR managers compared baseline dashboards to versions enriched with three XAI styles: important features, counterfactuals, and model criteria. Our results show that the dashboards used in practice do not explain AI results and even keep AI elements opaque. However, while adding XAI features improves subjective perceptions of helpfulness and trust among users with moderate or high AI literacy, it does not increase their objective understanding. It may even reduce accurate understanding, especially with complex explanations. Only overlays of important features significantly aided the interpretations of high-literacy users. Our findings highlight that the benefits of XAI in recruitment depend on users' AI literacy, emphasizing the need for tailored explanation strategies and targeted literacy training in HRM to ensure fair, transparent, and effective adoption of AI. KW - AI Literacy KW - Explainable AI KW - Recommender Systems KW - Human Resource Management KW - Recruitment KW - Explainable AI KW - Recommender System KW - Human Resource Management KW - Recruitment Y1 - 2025 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:0074-4046-X SN - 1613-0073 SP - 1 EP - 10 PB - CEUR ER - TY - CHAP A1 - Schauer, Sophie A1 - Simbeck, Katharina A1 - Kühn, Patricia T1 - Generative AI in Museums: Design Concepts for an Accessible Digital Mediation Station for a Renaissance Altar T2 - Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering N2 - Exploring the potential of artificial intelligence (AI) in museum settings, this study finds concepts for a digital and inclusive mediation station giving context to a Renaissance sandstone altar at the Staatliches Museum Schwerin. Today, many museum visitors have less theological knowledge and come from diverse, often non-Christian, backgrounds. Thus, understanding and appreciating historical artworks and artefacts becomes more difficult. Therefore, exploring ways to present cultural heritage in a manner that is understandable, playful, and inclusive, regardless of the audience’s origin or beliefs, is necessary. Through a collaborative week-long workshop involving communication design students, conceptual frameworks and prototype designs were iteratively developed and evaluated, laying the groundwork for further refinement and subsequent integration into the museum’s permanent exhibition. KW - Generative AI KW - Inclusion and Accessibility in Museums KW - Cultural Heritage KW - Diversity KW - Concept Design Y1 - 2025 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:523-22703 SP - 283 EP - 295 PB - Springer CY - Heidelberg ER - TY - CHAP A1 - Schauer, Sophie A1 - Katharina, Simbeck A1 - Fernsel, Linda T1 - The System Admin’s Perspective: A Discussion on AI in Education with LMS Admins T2 - International Conference on Information Technology Based Higher Education and Training (ITHET) N2 - Many higher education institutions (HEIs) are implementing or reviewing the implementation of predictive Learning Analytics to improve learning processes, identify students at risk or provide personalized learning paths. The responsibility for selecting, reviewing, implementing and supporting those systems falls on system administrators, an under-researched stakeholder group of Artificial Intelligence in higher education. In this paper, we summarize qualitative insights from a workshop with system administrators in German HEIs. We find that the system administrators are highly aware not only of system requirements but also of the needs of various shareholder groups such as institutional leadership, learners and educators and that they put high emphasis on ethical, transparent and compliant system use. We conclude that system administrators should be involvedmore in research on the use of technology in education and that AI systems used in education need to provide possibilities to sufficiently test the system, including anonymous yet realistic test scenarios and data. KW - Predictive Learning Analytics KW - Ethical Learning Analytics KW - Learning Management Systems KW - Moodle KW - AI audit Y1 - 2025 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:523-22759 SP - 1 EP - 5 PB - IEEE ER -