TY - JOUR A1 - Grabinger, Lisa A1 - Hauser, Florian A1 - Mottok, Jürgen T1 - On the perception of graph layouts JF - Journal of Software: Evolution and Process N2 - In the field of software engineering, graph-based models are used for a variety of applications. Usually, the layout of those graphs is determined at the discretion of the user. This article empirically investigates whether different layouts affect the comprehensibility or popularity of a graph and whether one can predict the perception of certain aspects in the graph using basic graphical laws from psychology (i.e., Gestalt principles). Data on three distinct layouts of one causal graph is collected from 29 subjects using eye tracking and a print questionnaire. The evaluation of the collected data suggests that the layout of a graph does matter and that the Gestalt principles are a valuable tool for assessing partial aspects of a layout. KW - causal graphs KW - eye tracking KW - gestalt principles KW - graph layouts KW - modeling languages Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-62973 N1 - Corresponding author: Lisa Grabinger PB - Wiley ER - TY - CHAP A1 - Grabinger, Lisa A1 - Hauser, Florian A1 - Mottok, Jürgen T1 - Accessing the Presentation of Causal Graphs and an Application of Gestalt Principles with Eye Tracking T2 - 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER 2022), 1st Workshop on Advances in Human-Centric Experiments in Software Engineering (HUMAN 2022): 15-18 March 2022, Honolulu, HI, USA N2 - The discipline of causal inference uses so-called causal graphs to model cause and effect relations of random variables. As those graphs only encode a relation structure there is no hard rule concerning their alignment. The present paper presents a study with the aim of working out the optimal alignment of causal graphs with respect to comprehensibility and interestingness. In addition, the study examines whether the central gestalt principles of psychology apply for causal graphs. Data from 29 participants is acquired by triangulating eye tracking with a questionnaire. The results of the study suggest that causal graphs should be aligned downwards. Moreover, the gestalt principles proximity, similarity and closure are shown to hold true for causal graphs. KW - causal graphs KW - gestalt principles KW - eye tracking Y1 - 2022 U6 - https://doi.org/10.1109/SANER53432.2022.00153 SP - 1278 EP - 1285 PB - IEEE ER - TY - CHAP A1 - Homann, Alexander A1 - Grabinger, Lisa A1 - Hauser, Florian A1 - Mottok, Jürgen T1 - An Eye Tracking Study on MISRA C Coding Guidelines T2 - ECSEE '23: Proceedings of the 5th European Conference on Software Engineering Education, Seeon/Germany, June 19 - 21, 2023 N2 - C is one of the most widely used programming languages - MISRA C is one of the most known sets of coding guidelines for C. This paper examines the usefulness and comprehensibility of the MISRA C:2012 guidelines in an eye tracking study. There, subjects encounter non-compliant code in four different code review settings: with no additional reference, with an actual MISRA C guideline, with a case-specific interpretation of a MISRA C guideline, and with a compliant version of the code. The data collected was analyzed not only in terms of the four presentation styles, but also by dividing the subjects into experience levels based on their semesters of study or years of work experience. Regarding the difference between actual and interpreted guidelines, we found that for interpreted guidelines the error detection rate is higher whereas the duration and frequency of visits to the guideline itself are mainly lower. This suggest that the actual guidelines are less useful and more difficult to understand. The former is contradicted by the subjects’ opinions: when surveyed, they rated the usefulness of the actual guidelines higher. Y1 - 2023 SN - 978-1-4503-9956-2 U6 - https://doi.org/10.1145/3593663.3593671 SP - 130 EP - 137 PB - ACM ER - TY - CHAP A1 - Grabinger, Lisa A1 - Hauser, Florian A1 - Mottok, Jürgen T1 - Evaluating Graph-based Modeling Languages T2 - Proceedings of the 5th European Conference on Software Engineering Education, Seeon/Germany, June 19 - 21, 2023 N2 - As humans, we tend to use models to describe reality. Modeling languages provide the formal frameworks for creating such models. Usually, the graphical design of individual model elements is based on subjective decisions; their suitability is determined at most by the prevalence of the modeling language. With other words: there is no objective way to compare different designs of model elements. The present paper addresses this issue: it introduces a systematic approach for evaluating the elements of graph-based modeling languages comprising 14 criteria – derived from standards, usability analyses, or the design theories ‘Physics of Notations’ and ‘Cognitive Dimensions of Notations’. The criteria come with measurement procedures and evaluation schemes based on reasoning, eye tracking, and questioning. The developed approach is demonstrated with a specific use case: three distinct sets of node elements for causal graphs are evaluated in an eye tracking study with 41 subjects. Y1 - 2023 SN - 978-1-4503-9956-2 U6 - https://doi.org/10.1145/3593663.3593664 SP - 120 EP - 129 PB - ACM ER - TY - CHAP A1 - Bittner, Dominik A1 - Hauser, Florian A1 - Nadimpalli, Vamsi Krishna A1 - Grabinger, Lisa A1 - Staufer, Susanne A1 - Mottok, Jürgen T1 - Towards Eye Tracking based Learning Style Identification T2 - Proceedings of the 5th European Conference on Software Engineering Education, Seeon/Germany, June 19 - 21, 2023 N2 - The dropout rate at universities has been very high for years. Thereby, the inexperience and lack of knowledge of students in dealing with individual learning paths in various courses of study plays a decisive role. Adaptive learning management systems are suitable countermeasures, in which learners’ learning styles are classified using questionnaires or computationally intensive algorithms before a learning path is suggested accordingly. In this paper, a study design for student learning style classification using eye tracking is presented. Furthermore, qualitative and quantitative analyses clarify certain relationships between students’ eye movements and learning styles. With the help of classification based on eye tracking, the filling out of questionnaires or the integration of computationally or cost-intensive algorithms can be made redundant in the future. Y1 - 2023 SN - 978-1-4503-9956-2 U6 - https://doi.org/10.1145/3593663.3593680 SP - 138 EP - 147 PB - ACM ER - TY - CHAP A1 - Hauser, Florian A1 - Grabinger, Lisa A1 - Mottok, Jürgen T1 - Something Short Gets Even Shorter: Adapting the LIST-K for the Use in an Online Learning Management System T2 - Proceedings of the 5th European Conference on Software Engineering Education, Seeon/Germany, June 19 - 21, 2023 N2 - This study examines how Klingsieck’s LIST-K questionnaire [22] can be shortened and adapted to the requirements of an online learning management system. In a study with 213 participants, the questionnaire is subjected to an exploitative factor analysis. In a next step, the results are evaluated in terms of their reliability. This process creates a modified factor structure for the LIST-K, comprising a total of eight factors. The reliability of the modified questionnaire is at an α of .770. The shortened version of the LIST-K questionnaire is currently being used on an experimental basis in different courses. Y1 - 2023 SN - 978-1-4503-9956-2 U6 - https://doi.org/10.1145/3593663.3593684 SP - 65 EP - 72 PB - ACM ER - TY - CHAP A1 - Hauser, Florian A1 - Grabinger, Lisa A1 - Mottok, Jürgen A1 - Jahn, Sabrina A1 - Nadimpalli, Vamsi Krishna T1 - The Expert’s View: Eye Movement Modeling Examples in Software Engineering Education T2 - Proceedings of the 5th European Conference on Software Engineering Education, Seeon/Germany, June 19 - 21, 2023 N2 - This study investigates the impact of eye movement modeling examples in Software Engineering education. Software Engineering is a highly visual domain. The daily tasks of a software engineer (e.g., formulating requirements, creating UML diagrams, or conducting a code review) require in many cases the use of certain visual strategies. Although these strategies can be found for experts, it has been observed in different eye tracking studies that students have difficulties in learning and applying them. To familiarize students with these visual strategies and to provide them with a better understanding for the cognitive processes involved, a total of seven eye movement modeling examples was created. The seven eye movement modeling examples cover relevant parts of an introductory Software Engineering lecture; they are focused on typical situations in which visual strategies are applied. The results of a questionnaire-based evaluation shows that students consider the eye movement modeling examples as useful, feel supported in their learning process, and would like to see more use of them in the Software Engineering lecture. Furthermore, the students suggested that eye movement modeling examples should also be used in other lectures. Y1 - 2023 SN - 978-1-4503-9956-2 U6 - https://doi.org/10.1145/3593663.3593683 SP - 148 EP - 152 PB - ACM ER - TY - CHAP A1 - Nadimpalli, Vamsi Krishna A1 - Hauser, Florian A1 - Bittner, Dominik A1 - Grabinger, Lisa A1 - Staufer, Susanne A1 - Mottok, Jürgen ED - Mottok, Jürgen T1 - Systematic Literature Review for the Use of AI Based Techniques in Adaptive Learning Management Systems T2 - Proceedings of the 5th European Conference on Software Engineering Education, Seeon/Germany, June 19 - 21, 2023 N2 - Nowadays, learning management systems are widely employed in all educational institutions to instruct students as a result of the increasing in online usage. Today’s learning management systems provide learning paths without personalizing them to the characteristics of the learner. Therefore, research these days is concentrated on employing AI-based strategies to personalize the systems. However, there are many different AI algorithms, making it challenging to determine which ones are most suited for taking into account the many different features of learner data and learning contents. This paper conducts a systematic literature review in order to discuss the AI-based methods that are frequently used to identify learner characteristics, organize the learning contents, recommend learning paths, and highlight their advantages and disadvantages. Y1 - 2023 SN - 978-1-4503-9956-2 U6 - https://doi.org/10.1145/3593663.3593681 SP - 83 EP - 92 PB - Association for Computing Machinery CY - New York ER - TY - CHAP A1 - Bugert, Flemming A1 - Grabinger, Lisa A1 - Bittner, Dominik A1 - Hauser, Florian A1 - Nadimpalli, Vamsi Krishna A1 - Staufer, Susanne A1 - Mottok, Jürgen T1 - Towards Learning Style Prediction based on Personality T2 - Proceedings of the 5th European Conference on Software Engineering Education, Seeon/Germany, June 19 - 21, 2023 N2 - This paper assesses the relation between personality, demographics, and learning style. Hence, data is collected from 200 participants using 1) the BFI-10 to obtain the participant’s expression of personality traits according to the five-factor model, 2) the ILS to determine the participant’s learning style according to Felder and Silverman, and 3) a demographic questionnaire. From the obtained data, we train and evaluate a Bayesian network. Using Bayesian statistics, we show that age and gender slightly influence personality and that demographics as well as personality have at least a minor effect on learning styles. We also discuss the limitations and future work of the presented approach. Y1 - 2023 SN - 978-1-4503-9956-2 U6 - https://doi.org/10.1145/3593663.3593682 SP - 48 EP - 55 PB - ACM ER - TY - JOUR A1 - Maier, Robert A1 - Grabinger, Lisa A1 - Urlhart, David A1 - Mottok, Jürgen T1 - Causal Models to Support Scenario-Based Testing of ADAS JF - IEEE Transactions on Intelligent Transportation Systems N2 - In modern vehicles, system complexity and technical capabilities are constantly growing. As a result, manufacturers and regulators are both increasingly challenged to ensure the reliability, safety, and intended behavior of these systems. With current methodologies, it is difficult to address the various interactions between vehicle components and environmental factors. However, model-based engineering offers a solution by allowing to abstract reality and enhancing communication among engineers and stakeholders. Applying this method requires a model format that is machine-processable, human-understandable, and mathematically sound. In addition, the model format needs to support probabilistic reasoning to account for incomplete data and knowledge about a problem domain. We propose structural causal models as a suitable framework for addressing these demands. In this article, we show how to combine data from different sources into an inferable causal model for an advanced driver-assistance system. We then consider the developed causal model for scenario-based testing to illustrate how a model-based approach can improve industrial system development processes. We conclude this paper by discussing the ongoing challenges to our approach and provide pointers for future work. KW - automated driving systems KW - Automation KW - Bayesian networks KW - Causal inference KW - Data models KW - ISO Standards KW - model-based testing KW - Safety KW - Task analysis KW - Testing KW - Vehicles Y1 - 2023 U6 - https://doi.org/10.1109/TITS.2023.3317475 SN - 1524-9050 SP - 1 EP - 17 PB - IEEE ER - TY - CHAP A1 - Nadimpalli, Vamsi Krishna A1 - Bugert, Flemmimg A1 - Bittner, Dominik A1 - Hauser, Florian A1 - Grabinger, Lisa A1 - Staufer, Susanne A1 - Mottok, Jürgen ED - Gómez Chova, Luis ED - González Martínez, Chelo ED - Lees, Joanna T1 - Towards personalized learning paths in adaptive learning management systems: bayesian modelling of psychological theories T2 - Proceedings of the 16th annual International Conference of Education, Research and Innovation (ICERI 2023), Seville, Spain, 11/13/2023 - 11/14/2023 N2 - In educational research, non-personalized learning content increases learners' cognitive load, causing them to lower their performance and sometimes drop out of the course. Personalizing learning content with learners’ unique characteristics, like learning styles, personality traits, and learning strategies, is being suggested to improve learners’ success. Several theories exist for assessing learners’ unique characteristics. By the end of 2020, 71 learning style theories have been formulated, and research has shown that combining multiple learning style theories to recommend learning paths yields better results. As of the end of 2022, there is no single research that demonstrates a relationship between the Index of Learning Styles (ILS) based Felder-Silverman learning style model (FSLSM) dimensions, Big Five (BFI-10) based personality traits, and the Learning strategies in studying (LIST-K) based learning strategies factors for personalizing learning content. In this paper, an innovative approach is proposed to estimate the relationship between these theories and map the corresponding learning elements to create personalized learning paths. Respective questionnaires were distributed to 297 higher education students for data collection. A three-step approach was formulated to estimate the relationship between the models. First, a literature search was conducted to find existing studies. Then, an expert interview was carried out with a group of one software engineering education research professor, three doctoral students, and two master’s students. Finally, the correlations between the students' questionnaire responses were calculated. To achieve this, a Bayesian Network was built with expert knowledge from the three-step approach, and the weights were learned from collected data. The probability of individual FSLSM learning style dimensions was estimated for a new test sample. Based on the literature, the learning elements were mapped to the respective FSLSM learning style dimensions and were initiated as learning paths to the learners. The next steps are proposed to extend this framework and dynamically recommend learning paths in real time. In addition, the individual levels of learning style dimensions, personality traits, and learning strategies can be considered to improve the recommendations. Further, using probabilities for mapping learning elements to learning styles can increase the chance of initiating multiple learning paths for an individual learner. Y1 - 2023 U6 - https://doi.org/10.21125/iceri.2023.1144 SP - 4593 EP - 4603 PB - IATED ER - TY - CHAP A1 - Staufer, Susanne A1 - Hauser, Florian A1 - Grabinger, Lisa A1 - Bittner, Dominik A1 - Nadimpalli, Vamsi Krishna A1 - Mottok, Jürgen ED - Gómez Chova, Luis ED - González Martínez, Chelo ED - Lees, Joanna T1 - Learning elements in online learning management systems T2 - Proceedings of the 16th annual International Conference of Education, Research and Innovation (ICERI 2023), Seville, Spain, 11/13/2023 - 11/14/2023 Y1 - 2023 U6 - https://doi.org/10.21125/iceri.2023.0815 SP - 3121 EP - 3130 PB - IATED ER - TY - CHAP A1 - Maier, Robert A1 - Grabinger, Lisa A1 - Urlhart, David A1 - Mottok, Jürgen ED - Seguin, Christel ED - Zeller, Marc ED - Prosvirnova, Tatiana T1 - Towards Causal Model-Based Engineering in Automotive System Safety T2 - Model-Based Safety and Assessment, 8th International Symposium, IMBSA 2022: Munich, Germany, September 5–7, 2022, Proceedings N2 - Engineering is based on the understanding of causes and effects. Thus, causality should also guide the safety assessment of complex systems such as autonomous driving cars. To ensure the safety of the intended functionality of these systems, normative regulations like ISO 21448 recommend scenario-based testing. An important task here is to identify critical scenarios, so-called edge and corner cases. Data-driven approaches to this task (e.g. based on machine learning) cannot adequately address a constantly changing operational design domain. Model-based approaches offer a remedy – they allow including different sources of knowledge (e.g. data, human experts) into safety considerations. With this paper, we outline a novel approach for ensuring automotive system safety. We propose to use structural causal models as a probabilistic modelling language to combine knowledge about an open-context environment from different sources. Based on these models, we investigate parameter configurations that are candidates for critical scenarios. In this paper, we first discuss some aspects of scenario-based testing. We then provide an informal introduction to causal models and relate their development lifecycle to the established V-model. Finally, we outline a generic workflow for using causal models to identify critical scenarios and highlight some challenges that arise in the process. Y1 - 2022 U6 - https://doi.org/10.1007/978-3-031-15842-1_9 SP - 116 EP - 129 PB - Springer Nature CY - Cham ER - TY - CHAP A1 - Hauser, Florian A1 - Grabinger, Lisa A1 - Mottok, Jürgen A1 - Gruber, Hans T1 - Visual Expertise in Code Reviews: Using Holistic Models of Image Perception to Analyze and Interpret Eye Movements T2 - ETRA '23: 2023 Symposium on Eye Tracking Research and Applications, Tubingen Germany 30 May 2023- 2 June 2023 N2 - This study uses holistic models of image perception to analyze and interpret eye movements during a code review. 23 participants (15 novices and 8 experts) take part in the experiment. The subjects’ task is to review six short code examples in C programming language and identify possible errors. During the experiment, their eye movements are recorded by an SMI 250 REDmobile. Additional data is collected through questionnaires and retrospective interviews. The results implicate that holistic models of image perception provide a suitable theoretical background for the analysis and interpretation of eye movements during code reviews. The assumptions of these models are particularly evident for expert programmers. Their approach can be divided into different phases with characteristic eye movement patterns. It is best described as switching between scans of the code example (global viewing) and the detailed examination of errors (focal viewing). Y1 - 2023 U6 - https://doi.org/10.1145/3588015.3589189 SP - 1 EP - 7 PB - ACM ER - TY - CHAP A1 - Hauser, Florian A1 - Staufer, Susanne A1 - Grabinger, Lisa A1 - Röhrl, Simon A1 - Mottok, Jürgen ED - Gómez Chova, Luis ED - González Martínez, Chelo ED - Lees, Joanna T1 - On the analysis of student learning strategies: using the LIST-K questionnaire ro generate ai-based individualized learning paths T2 - Proceedings of the 16th annual International Conference of Education, Research and Innovation (ICERI 2023), Seville, Spain, 11/13/2023 - 11/14/2023 N2 - This paper presents the results of a data collection with the LIST-K questionnaire. This questionnaire measures students’ learning strategies and shows which strategies are particularly dominant or rather weak. Learning strategies have long been a major area of research in educational science and psychology. In these disciplines, learning strategies are understood as intentional behaviors and cognitive skills that learners employ to effectively complete learning tasks, by selecting, acquiring, organizing, and integrating information into their existing knowledge for long-term retention. The LIST-K, developed by Klingsieck in 2018, was chosen for accessing learning strategies due to its thematic suitability, widespread use, and test economy. It covers a total of four main categories (i.e., cognitive strategies, metacognitive strategies, management of internal resources, and management of external resources), each of which are subdivided into further subscales. With a total of 39 items answered via a 5-step Likert scale, the LIST-K can cover the topic relatively comprehensively and at the same time be completed in a reasonable amount of time of approximately 10 minutes. The LIST-K was used as part of a combined data collection along with other questionnaires on their personal data, their preferences regarding certain learning elements, their learning style (i.e. the ILS), and personality (i.e. the BFI-10). A total of 207 students from different study programs participated via an online survey created using the survey tool "LimeSurvey". Participation in the study was voluntary, anonymously, and in compliance with the GDPR. Overall, the results of the LIST-K show that students are willing to work intensively on relevant topics intensively and to perform beyond the requirements of the course seeking additional learning material. At the same time, however, it is apparent that the organization of their own learning process could still be improved. For example, students start repeating content too late (mean=2.70; SD=0.92) and do not set goals for themselves and do not create a learning plan (mean=3.19; SD=0.90). They also learn without a schedule (mean=2.23; SD=0.97) and miss opportunities to learn together with other students (mean=3.17; SD=0.94). The findings of the data collection will be used to create an AI-based adaptive learning management system that will create individualized learning paths for students in their respective courses. From the results of the LIST-K, it appears that the adaptive learning management system should primarily support organizational aspects of student learning. Even small impulses (an individual schedule of when to learn what or a hierarchical structuring of the learning material) could help students to complete their courses more successfully and improve their learning. Y1 - 2023 U6 - https://doi.org/10.21125/iceri.2023.1147 SP - 4611 EP - 4620 PB - IATED ER - TY - CHAP A1 - Bittner, Dominik A1 - Ezer, Timur A1 - Grabinger, Lisa A1 - Hauser, Florian A1 - Mottok, Jürgen ED - Gómez Chova, Luis ED - González Martínez, Chelo ED - Lees, Joanna T1 - Unveiling the secrets of learning styles: decoding eye movements via machine learning T2 - Proceedings of the 16th annual International Conference of Education, Research and Innovation (ICERI 2023), Seville, Spain, 11/13/2023 - 11/14/2023 N2 - Universities are faced with a rising number of dropouts in recent years. This is largely due to students' limited capability of finding individual learning paths through various course materials. However, a possible solution to this problem is the introduction of adaptive learning management systems, which recommend tailored learning paths to students – based on their individual learning styles. For the classification of learning styles, the most commonly used methods are questionnaires and learning analytics. Nevertheless, both methods are prone to errors: questionnaires may give superficial answers due to lack of time or motivation, while learning analytics do not reflect offline learning behavior. This paper proposes an alternative approach to classify students' learning styles by integrating eye tracking in combination with Machine Learning (ML) algorithms. Incorporating eye tracking technology into the classification process eliminates the potential problems arising from questionnaires or learning analytics by providing a more objective and detailed analysis of the subject's behavior. Moreover, this approach allows for a deeper understanding of subconscious processes and provides valuable insights into the individualized learning preferences of students. In order to demonstrate this approach, an eye tracking study is conducted with 117 participants using the Tobii Pro Fusion. Using qualitative and quantitative analyses, certain patterns in the subjects' gaze behavior are assigned to their learning styles given by the validated Index of Learning Styles (ILS) questionnaire. In short, this paper presents an innovative solution to the challenges associated with classifying students' learning styles. By combining eye tracking data with ML algorithms, an accurate and insightful understanding of students' individual learning paths can be achieved, ultimately leading to improved educational outcomes and reduced dropout rates. Y1 - 2023 SN - 978-84-09-55942-8 U6 - https://doi.org/10.21125/iceri.2023.1291 SP - 5153 EP - 5162 PB - IATED ER - TY - CHAP A1 - Ezer, Timur A1 - Greiner, Matthias A1 - Grabinger, Lisa A1 - Hauser, Florian A1 - Mottok, Jürgen ED - Gómez Chova, Luis ED - González Martínez, Chelo ED - Lees, Joanna T1 - Eye tracking al technology in education; data quality analysis and improvements T2 - Proceedings of the 16th annual International Conference of Education, Research and Innovation (ICERI 2023), Seville, Spain, 11/13/2023 - 11/14/2023 N2 - Eye tracking has proven to be a powerful tool in a variety of empirical research areas; hence, it is steadily gaining attention. Driven by the expanding frontiers of Artificial Intelligence and its potential for data analysis, eye tracking technology offers promising applications in diverse fields, from usability research to cognitive research. The education sector in particular can benefit from the increased use of eye tracking technology - both indirectly, for example by studying the differences in gaze patterns between experts and novices to identify promising strategies, and directly by using the technology itself to teach in future classrooms. As with any empirical method, the results depend directly on the quality of the data collected. That raises the question of which parameters educators or researchers can influence to maximize the data quality of an eye tracker. This is the starting point of the present work: In an empirical study of eye tracking as an (educational) technology, we systematically examine factors that influence the data quality, such as illumination, sampling frequency, and head orientation - parameters that can be varied without much additional effort in everyday classroom or research use - using two human subjects, an artificial face, and the Tobii Pro Spectrum. We rely on metrics derived from the raw gaze data, such as accuracy or precision, to measure data quality. The obtained results derive practical advice for educators and researchers, such as using the lowest sampling frequency appropriate for a certain purpose. Thereby, this research fills a gap in the current understanding of eye tracker performance and, by offering best practices, enables researchers or teachers to produce data of the highest quality possible and therefore best results when using eye trackers in laboratories or future classrooms. Y1 - 2023 U6 - https://doi.org/10.21125/iceri.2023.1127 SP - 4500 EP - 4509 PB - IATED ER -