TY - CHAP A1 - Haug, Jim A1 - Sapsai, Iryna A1 - Hock, Isabell A1 - Abke, Jörg A1 - Hagel, Georg ED - Haug, Jim T1 - EVALUATING AN AI-BASED ADAPTIVE LEARNING SYSTEM: GOALS, METHODS AND INITIAL RESULTS T2 - 16th International Conference on Education and New Learning Technologies, 1-3 July, 2024, Palma, Spain N2 - The aim of this paper is to describe the evaluation process and findings of an AI-based Adaptive Learning System for the Computer Science discipline at two different German universities and discuss an array of methods in regard to assessing such a system. The primary objectives have been twofold: firstly, to examine the reception of selected learning elements, which were conceptually outlined based on relevant literature, among the student body; and secondly, to investigate the efficacy of individualized adaptive learning paths. These paths were generated by employing a variety of algorithms to analyze students learning style tendencies, with a particular emphasis on adaptive navigational techniques. The used algorithms encompassed a modified version of a literature based adaptive mechanism, an Ant-Colony-Algorithm and a Genetic Algorithm, alongside a lecturer-recommended learning path for a non-adaptive comparison. While the system suggested suitable learning paths based on student data, it never forced the individuals to give up their self-directed learning. The evaluation criteria revolved around the evolution of student motivation, interest levels, and knowledge acquisition during the time they spent working in the system. The evaluation sought to facilitate comparative analyses and assess algorithmic fitness for proficient learning path generation. The methods included both quantitative and qualitative approaches to gather data, seeking to strike a balance between being student-friendly and scientifically informative. They ranged from Likert Scale self-assessments to screen and video observations with retrospective interviews. Since the purpose of adaptive learning systems is intertwined with personalized learning it seems imperative to already take the preferences and opinions of students into account while the system is still in development. This complexity underscores the challenge of evaluating such systems, as significant constraints on student choice - though simplifying evaluation - directly oppose the ethos of individualized, self-directed learning. Initial findings suggest that the underlying theoretical considerations on sequencing and structuring of learning elements are confirmed, coupled with providing adequate flexibility to meet diverse learning needs. Cross-site evaluation of the literature-based learning elements indicated a high comprehensibility and positive student ratings. While significant positive trends were observed regarding knowledge acquisition, they cannot be definitively attributed to a specific method of learning path generation. Motivation and interest analyses show no significant differences among learning path types, albeit heavily limited by sample size. Similarly, emotion measurements, though limited, hint at positive impacts from HASKI system use. Despite limitations, early indications suggest student acceptance and potential effectiveness of learning paths, highlighting the need for larger sample sizes for validation and expansion. Ensuring alignment with student needs and user-friendly design are crucial considerations. KW - Adaptive learning environment, tertiary education, personalized e-learning, evaluation. KW - E-Learning KW - Evaluation KW - Künstliche Intelligenz Y1 - 2024 UR - https://library.iated.org/view/HAUG2024EVA U6 - https://doi.org/10.21125/edulearn.2024.0834 VL - 2024 IS - EDULEARN24 Proceedings SP - 3157 EP - 3166 ER - TY - JOUR A1 - Sapsai, Iryna A1 - Haug, Jim A1 - Abke, Jörg A1 - Hagel, Georg T1 - IDENTIFYING STUDENT EMOTIONS IN AN ADAPTIVE LEARNING SYSTEM WITH A BAYESIAN NETWORK MODEL JF - IATED Digital Library Home N2 - Analyzing scientific literature on the impact of emotions in e-learning highlights the importance of assessing students' emotional states and their role during online learning processes. This assessment often relies on physiological sensors, which may not be accessible to all institutions or acceptable to all students. Therefore, detecting students' emotional states in an online learning environment remains a complex challenge requiring a fundamental understanding of the role of emotions in such environments. This paper explores the potential benefits of using self-reported surveys to identify students' affective states and improve their learning experience by responding to specific emotional states during their learning progress. Our objectives include identifying emotional states through surveys, pinpointing frequently selected emotion-descriptive words, analyzing survey results, and developing a Bayesian Network (BN) model for automatic emotional state analysis. This BN model aims to facilitate real-time interactions within an adaptive learning system by autonomously assessing students' emotions without direct educator intervention. We present findings from an emotion identification survey conducted among engineering students in an e-learning course provided in an adaptive learning environment. Practically, we plan to implement the BN model within the aforementioned environment to interact with students through real-time pop-up messages, identifying and responding to their emotional states. Its generic character structure allows for changes in the set of analyzed words, ensuring its broad applicability. The challenges of identifying emotions during different online learning activities and the question of the optimal word selection remain for future exploration. Effective identification of students' emotional states is expected to enhance online learning, improving emotional well-being, motivation, and individual success. KW - e-learning, Emotion Identification, Automatic Detection Techniques, Bayesian Networks. KW - Bayes-Netz KW - E-Learning Y1 - 2024 UR - https://library.iated.org/view/SAPSAI2024IDE U6 - https://doi.org/10.21125/iceri.2024.1192 VL - 2024 IS - ICERI2024 Proceedings SP - 4829 EP - 4837 ER - TY - CHAP A1 - Abke, Jörg A1 - Brune, Philipp A1 - Haupt, Wolfram A1 - Hagel, Georg A1 - Landes, Dieter A1 - Mottok, Jürgen T1 - EVELIN – ein Forschungsprojekt zur systematischen Verbesserung des Lernens von Software Engineering T2 - Tagungsband Embedded Software Engineering-Kongress 2012 N2 - Software bestimmt unseren Alltag, ist aber abstrakt und physisch nicht greifbar. Wegen ihres hohen Stellenwerts ist eine gute Software-Engineering-Ausbildung von zentraler Bedeutung. Eine hohe Technologiekompetenz der Softwareentwickler ist allein kein Garant für qualitativ hochwertige Software, entscheidend sind auch Soft Skills wie Kommunikations- und Teamfähigkeit der beteiligten Ingenieure und Informatiker. Neben dem hohen Abstraktionsgrad stellen also auch weiche Faktoren die Ausbildung im Software Engineering vor große Herausforderungen. Didaktische Ansätze zur Vermittlung von Software Engineering müssen hinsichtlich beider Anforderungen systematisch weiterentwickelt werden. Die „klassische“ Ausbildung mit Elementen des Frontalunterrichts ist durch Methodenarrangements zu ergänzen, die die Prozesse des Lernens und Lehrens fokussieren. Im Forschungsprojekt EVELIN („Experimentelle Verbesserung des Lernens von Software Engineering“) arbeiten seit Anfang 2012 sechs Hochschulen an der systematischen kompetenzorientierten Weiterentwicklung der Software-Engineering- Ausbildung. Der Vortrag skizziert Ziele und den aktuellen Stand des Forschungsprojektes EVELIN. KW - Software Engineering KW - Forschungsprojekt Y1 - 2012 SP - 653 EP - 658 ER - TY - CHAP A1 - Normann, Marc A1 - Haug, Jim A1 - Valencia, Yeimy A1 - Abke, Jörg A1 - Hagel, Georg ED - Mottock, Jürgen ED - Hagel, Georg T1 - Adaptive Learning Path Sequencing Based on Learning Styles within N-dimensional Spaces T2 - ECSEE '23: Proceedings of the 5th European Conference on Software Engineering Education N2 - Planning adaptive learning paths for students’ progress throughout a course can be a challenging task, although it can be helpful for their learning progress. Within the HASKI-System, students should be able to get their own, personalized learning paths. In this paper, we present an approach towards the learning path sequencing problem. This idea is based on a novel proposal for arranging learning objects in a multi-dimensional space, bringing the relationship and similarities of these objects into a new relationship. We show, that we can use both, the Ant Colony Optimization Algorithm and the Genetic Algorithm with the idea of the Traveling-Salesman-Problem and get results, that are comparable with a proposed literature-based adaption mechanism. Nevertheless, the learning paths are all personalized based on the Felder & Silverman Learning Style Model and the hyperspace model will allow us later on to include more dimensions for other influencing factors. KW - Learning Path Sequencing KW - Adaptive Learning Path KW - Ant Colony KW - Genetic Algorithm KW - Adaptive Learning Environment KW - Lernstil KW - Lernerfolg Y1 - 2023 UR - https://dl.acm.org/doi/abs/10.1145/3593663.3593676 U6 - https://doi.org/https://doi.org/10.1145/3593663.3593676 VL - 2023 SP - 56 EP - 64 ER - TY - CHAP A1 - Klopp, Marco A1 - Dörringer, Antonia A1 - Eigler, Tobias A1 - Bartel, Paula A1 - Hochstetter, Marvin A1 - Weishaupt, Andreas A1 - Geirhos, Philipp A1 - Abke, Jörg A1 - Hagel, Georg A1 - Elsebach, Jens A1 - Rossmann, Raphael ED - Mottok, Jürgen T1 - Development of an Authoring Tool for the Creation of Individual 3D Game-Based Learning Environments T2 - ECSEE '23: Proceedings of the 5th European Conference on Software Engineering Education N2 - Game-based learning in general and serious games in particular have a promising potential in higher education. In this article we going to show the capability of serious games with regard to current challenges in higher education. The focus of this article is the presentation of the AdLer authoring tool, which offers lecturers the possibility to design and generate virtual 3D learning environments in which students can interact with learning content according to the principles of game-based learning. KW - 3D Learning Environment KW - Authoring Tool KW - Game-based Learning KW - Serious Games KW - Lernspiel KW - Dreidimensionale Computergrafik Y1 - 2023 UR - https://dl.acm.org/doi/proceedings/10.1145/3593663 SN - 978-1-4503-9956-2 VL - 2023 IS - 5. SP - 204 EP - 209 PB - Association for Computing Machinery (ACM) CY - New York, NY, United States ER - TY - CHAP A1 - Dörringer, Antonia A1 - Klopp, Marco A1 - Schaab, Lukas A1 - Hochstetter, Marvin A1 - Glaab, Daniel A1 - Bartel, Paula A1 - Abke, Jörg A1 - Elsebach, Jens A1 - Rossmann, Raphael A1 - Hagel, Georg ED - Röpke, René ED - Schroeder, Ulrike T1 - AdLer: 3D-Lernumgebung für Studierende T2 - 21. Fachtagung Bildungstechnologien (DELFI) KW - Serious Games KW - 3D-Lernumgebung KW - Game-based Learning KW - Lernspiel KW - Dreidimensionale Computergrafik Y1 - 2023 U6 - https://doi.org/10.18420/delfi2023-41 VL - 2023 SP - 251 EP - 252 ER - TY - CHAP A1 - Manz, Julian A1 - Abke, Jörg A1 - Hagel, Georg ED - Mottok, Jürgen ED - Hagel, Georg T1 - Automated Tracking of User Interactions in Web-Based Adaptive Learning for Software Engineering T2 - ECSEE '25: Proceedings of the 6th European Conference on Software Engineering Education N2 - This paper explores the automation of generating and dispatching Experience API (xAPI) statements for comprehensive tracking of user interactions in e-learning environments. It introduces the react-xapi-wrapper library, an extension of the xAPI JavaScript library designed for use in web applications. Key aspects discussed include the library’s features, its integration into a web-based adaptive learning system (ALS) for software engineering, and the custom verbs used. The goal is to reduce implementation effort for tutors and developers while taking advantage of xAPI’s interoperability, scalability, and ability to track student learning activities and behaviors, laying the foundation for more responsive and personalized learning experiences. KW - Adaptive Learning System KW - Educational Technology KW - Experience API KW - Learning Analytics KW - Learning Record Store KW - Personalized Learning KW - Software Engineering Education KW - User Interaction Tracking KW - Computerunterstütztes Lernen KW - E-Learning Y1 - 2025 SN - 979-8-4007-1282-1 U6 - https://doi.org/10.1145/3723010.3723020 SP - 180 EP - 184 PB - Association for Computing Machinery CY - New York, NY, USA ER - TY - CHAP A1 - Fischer, David Vincent A1 - Haug, Jim A1 - Schoppel, Paul A1 - Abke, Jörg A1 - Becker, Matthias A1 - Hagel, Georg T1 - Evaluation of a Node-based Automatic Short Answer Tool “NodeGrade” T2 - Proceedings of the 6th European Conference on Software Engineering Education N2 - NodeGrade tries to provide a suitable solution for the problem of time-intensive short answer grading. This research focuses simultaneously on performance, functionality and user experience, which is underlined by a triangulated approach. The evaluation results show comparable performance of NodeGrade on public datasets, even outperforming GPT-4 on the SemEval 2013 Task 7. Matching of NodeGrade’s output with multiple human expert raters reveals some weaknesses regarding cases at the lower and upper boundary. In terms of user experience, the interviewed and observed students recognized both positive facets, like better learning support and helpful feedback, and negative sides, including technical limitations and lack of transparency. Overall, NodeGrade promises high potential for further practical use and testing in the field of software engineering education and automatic short answer grading. KW - Software Engineering KW - Hochschuldidaktik Y1 - 2025 U6 - https://doi.org/https://dl.acm.org/doi/10.1145/3723010.3723021 SP - 20 EP - 29 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Schoppel, Paul A1 - Haug, Jim A1 - Manz, Julian A1 - Bigler, Dimitri A1 - Hock, Isabell A1 - Abke, Jörg A1 - Hagel, Georg T1 - METHODICAL APPROACH FOR ANALYZING LEARNING PATH FITNESS IN AN AI-BASED ADAPTIVE LEARNING SYSTEM T2 - EDULEARN Proceedings N2 - Learning paths are a cornerstone of many adaptive learning systems, particularly those focusing on adaptive navigational techniques. Evaluating and analyzing these paths is therefore crucial to ensure they effectively support both learners and instructors. For eLearning this process must be highly scalable despite minimal oversight and little to no control over learners’ behavior. Consequently, learning path evaluation should be automated, user-friendly, and precise. However, current research on this topic often emphasizes simulations, performance metrics, or mathematical models, without fully considering the broader, learner-centered aspects necessary for meaningful adaptation. The authors prior findings also indicate that approaches to assessing the suitability of learning paths must be optimized. To address these gaps, this paper presents a potential methodological approach for comprehensive learning path evaluation, aiming to enhance both the precision of adaptive learning systems and the overall learning experience. Three different algorithms, derived from learning style tendencies and a lecturer recommendation, were analyzed as an illustrative example, although the method itself is not constrained by the form or data basis of these algorithms. The adaptive learning system utilizes various measures to gauge the suitability of a learning path, all gathered through real-time learner feedback. These measures include the correlation between students’ preferred path and each algorithm, referred to as it’s fitness, the alignment between students’ actual adherence to a generated path and their own perception of their study behavior as well as their satisfaction with the path, and the connection between algorithm fitness and both actual performance and perceived performance. To collect data, students were asked to create their own preferred learning paths by digitally arranging the provided learning elements after receiving an introduction to the respective categories. Once they had completed a topic with a generated learning path, they rated their satisfaction with it and indicated whether they had followed its sequence. They also estimated whether their knowledge level had changed. Learning analytics were then employed to compare these self-reports with students’ actual study behavior. Performance was measured using a rating system, while Spearman’s Rho and Kendall’s Tau served as the main correlation metrics for data analysis. The results indicate that all three algorithms produce paths more closely aligned with students’ preferred learning paths than the lecturer recommendation, although no single algorithm demonstrated clear dominance. Student satisfaction showed some correlation with the fitness of the generated learning path. Additionally, student ratings appeared to have a slight positive correlation with learning path fitness, whereas self-perceived performance showed no discernible difference. Analysis of the link between actual student behavior and their feedback suggested that students were not reliable in judging whether or not they had followed a learning path. These findings are consistent with the authors earlier work suggesting the potential effectiveness of the learning path algorithms examined, thus supporting this new methodological approach to analyzing learning paths. The study also provided valuable insights for further development; however, its limited sample size remains a challenge for validation. KW - E-Learning KW - Unterrichtsmethode Y1 - 2025 SN - 978-84-09-74218-9 U6 - https://doi.org/https://doi.org/10.21125/edulearn.2025.2430 SN - 2340-1117 VL - 1 SP - 9438 EP - 9446 PB - IATED ER -