TY - CHAP A1 - Kiesler, Natalie A1 - Opel, Simone A1 - Thorbrügge, Carsten T1 - With Great Power Comes Great Responsibility - Integrating Data Ethics into Computing Education T2 - Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 N2 - Most computing students enter the industry once they graduate. As future software engineers, they will be in powerful positions, making decisions that impact their personal lives, others, and society. Thus, preparing graduates for their careers is crucial by addressing ethical considerations, decision problems, and other concepts related to morals, values, and legal aspects (e.g., data protection, privacy, security, etc.) as part of computing curricula. In this paper, we propose the integration of data ethics into computing programs and provide a framework for an ethics module, including relevant competency-based learning objectives. The proposed module is based on a curricular analysis of all 71 German data science degree programs focusing on ethics courses. The course contents and competency goals were analyzed and classified based on their cognitive complexity. As the results proved the lack of competency- based learning outcomes, we designed observable competency goals, meaning knowledge, skills, and dispositions taken in the context of a task. In addition, we provide suggestions for contents, pedagogical instructions, and assessments in such a course. The proposed module serves as a first draft and resource to support other educators aiming to design such a course and who are willing to integrate it into computing curricula. KW - ethics, moral, values, data science, computing, curricula, competencies, higher education Y1 - 2024 U6 - https://doi.org/10.1145/3649217.3653637 SP - 471 EP - 477 PB - ACM CY - New York, NY, USA ER - TY - BOOK A1 - Kiesler, Natalie T1 - Modeling Programming Competency BT - A Qualitative Analysis N2 - This book covers a qualitative study on the programming competencies of novice learners in higher education. To be precise, the book investigates the expected programming competencies within basic programming education at universities and the extent to which the Computer Science curricula fail to provide transparent, observable learning outcomes and assessable competencies. The study analyzes empirical data on 35 exemplary universities' curricula and interviews with experts in the field. The book covers research desiderata, research design and methodology, an in-depth data analysis, and a presentation and discussion of results in the context of programming education. Addressing programming competency in such great detail is essential due to the increasing relevance of computing in today’s society and the need for competent programmers who will help shape our future. Although programming is a core tier of computing and many related disciplines, learning how to program can be challenging in higher education, and many students fail in introductory programming. The book aims to understand what programming means, what programming competency encompasses, and what teachers expect of novice learners. In addition, it illustrates the cognitive complexity of programming as an advanced competency, including knowledge, skills, and dispositions in context. So, the purpose is to communicate the breadth and depth of programming competency to educators and learners of programming, including institutions, curriculum designers, and accreditation bodies. Moreover, the book’s goal is to represent how a qualitative research methodology can be applied in the context of computing education research, as the qualitative research paradigm is still an exception in computing education research. The book provides new insights into programming competency. It outlines the components of programming competencies in terms of knowledge, skills, and dispositions and their cognitive complexity according to the CC2020 computing curricula and the Anderson-Krathwohl taxonomy of the cognitive domain. These insights are essential as programming constitutes one of the most relevant competencies in all computing study programs. In addition, being able to program describes the capability of solving problems, which is also a core competency in today’s increasingly digitalized society. In particular, the book reveals the great relevance of dispositions and other competency components in programming education, which curricula currently fail to recognize and specify. In addition, the book outlines the resulting implications for higher education institutions, educators, and student expectations. Yet another result of interest to graduate students is the multi-method study design that allows for the triangulation of data and results. Y1 - 2024 SN - 9783031471476 U6 - https://doi.org/10.1007/978-3-031-47148-3 PB - Springer International Publishing CY - Cham ER - TY - CHAP A1 - Grande, Virginia A1 - Kiesler, Natalie A1 - Francisco R., María Andreína T1 - Student Perspectives on Using a Large Language Model (LLM) for an Assignment on Professional Ethics T2 - Proceedings of the 2024 on Innovation and Technology in Computer Science Education Vol. 1 N2 - The advent of Large Language Models (LLMs) started a serious discussion among educators on how LLMs would affect, e.g., curricula, assessments, and students' competencies. Generative AI and LLMs also raised ethical questions and concerns for computing educators and professionals. This experience report presents an assignment within a course on professional competencies, including some related to ethics, that computing master's students need in their careers. For the assignment, student groups discussed the ethical process by Lennerfors et al. by analyzing a case: a fictional researcher considers whether to attend the real CHI 2024 conference in Hawaii. The tasks were (1) to participate in in-class discussions on the case, (2) to use an LLM of their choice as a discussion partner for said case, and (3) to document both discussions, reflecting on their use of the LLM. Students reported positive experiences with the LLM as a way to increase their knowledge and understanding, although some identified limitations. The LLM provided a wider set of options for action in the studied case, including unfeasible ones. The LLM would not select a course of action, so students had to choose themselves, which they saw as coherent. From the educators' perspective, there is a need for more instruction for students using LLMs: some students did not perceive the tools as such but rather as an authoritative knowledge base. Therefore, this work has implications for educators considering the use of LLMs as discussion partners or tools to practice critical thinking, especially in computing ethics education. KW - Large Language Models, LLMs, ChatGPT, ethics, student perspec- tive, experience report Y1 - 2024 U6 - https://doi.org/10.1145/3649217.3653624 SP - 478 EP - 484 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Barendsen, Erik A1 - Lonati, Violetta A1 - Quille, Keith A1 - Altin, Rukiye A1 - Divitini, Monica A1 - Hooshangi, Sara A1 - Karnalim, Oscar A1 - Kiesler, Natalie A1 - Melton, Madison A1 - Suero Montero, Calkin A1 - Morpurgo, Anna T1 - AI in and for K-12 Informatics Education. Life after Generative AI. T2 - Proceedings of the 2024 on ACM Virtual Global Computing Education Conference Vol. 2 N2 - The use and adoption of Generative AI (GenAI) has revolutionised various sectors, including computing education. However, this narrow focus comes at a cost to the wider AI in and for educational research. This working group aims to explore current trends and explore multiple sources of information to identify areas of AI research in K-12 informatics education that are being underserved but needed in the post-GenAI AI era. Our research focuses on three areas: curriculum, teacher-professional learning and policy. The denouement of this aims to identify trends and shortfalls for AI in and for K-12 informatics education. We will systematically review the current literature to identify themes and emerging trends in AI education at K-12. This will be done under two facets, curricula and teacher-professional learning. In addition, we will conduct interviews and surveys with educators and AI experts. Next, we will examine the current policy (such as the European AI Act, and Euro- pean Commission guidelines on the use of AI and data in education and training as well as international counterparts). Policies are often developed by both educators and experts in the domain, thus providing a source of topics or areas that may be added to our findings. Finally, by synthesising insights from educators, AI experts, and policymakers, as well as the literature and policy, our working group seeks to highlight possible future trends and shortfalls. KW - AI, GenAI, Generative AI, K-12, Informatics, Curricula Y1 - 2024 U6 - https://doi.org/10.1145/3649409.3691073 SP - 279 EP - 280 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Prather, James A1 - Leinonen, Juho A1 - Kiesler, Natalie A1 - Benario, Jamie Gorson A1 - Lau, Sam A1 - MacNeil, Stephen A1 - Norouzi, Narges A1 - Opel, Simone A1 - Pettit, Virginia A1 - Porter, Leo A1 - Reeves, Brent N. A1 - Savelka, Jaromir A1 - Smith, David H. A1 - Strickroth, Sven A1 - Zingaro, Daniel T1 - How Instructors Incorporate Generative AI into Teaching Computing T2 - Proceedings of the 2024 on Innovation and Technology in Computer Science Education Vol. 2 N2 - Generative AI (GenAI) has seen great advancements in the past two years and the conversation around adoption is increasing. Widely available GenAI tools are disrupting classroom practices as they can write and explain code with minimal student prompting. While most acknowledge that there is no way to stop students from using such tools, a consensus has yet to form on how students should use them if they choose to do so. At the same time, researchers have begun to introduce new pedagogical tools that integrate GenAI into computing curricula. These new tools offer students personalized help or attempt to teach prompting skills without undercutting code comprehension. This working group aims to detail the current landscape of education-focused GenAI tools and teaching approaches, present gaps where new tools or approaches could appear, identify good practice-examples, and provide a guide for instructors to utilize GenAI as they continue to adapt to this new era. Y1 - 2024 U6 - https://doi.org/10.1145/3649405.3659534 SP - 771 EP - 772 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Kiesler, Natalie A1 - Impagliazzo, John A1 - Biernacka, Katarzyna A1 - Kapoor, Amanpreet A1 - Kazmi, Zain A1 - Ramagoni, Sujeeth Goud A1 - Sane, Aamod A1 - Tran, Keith A1 - Taneja, Shubbhi A1 - Wu, Zihan T1 - Where's the Data? Finding and Reusing Datasets in Computing Education T2 - Working Group Reports on 2023 ACM Conference on Global Computing Education N2 - Computing education research (CER) is a rapidly advancing discipline, offering vast potential for data-driven, secondary research or replication studies. Although gathering and analyzing data for research seem straightforward, making research data publicly available to the community remains a challenge. Likewise, finding and reusing high-quality, prominent, and well-documented research data proves to be a daunting task. In this working group paper, the authors present their search for available datasets in the CER context (e.g., in databases and repositories). The available datasets are further analyzed using a newly developed metadata scheme and presented to the community as a resource. The second component of this work is a summary of the community’s perspective and concerns on publishing their research data, which has been gathered through a survey among 52 computing education researchers. Based on this status quo, this report presents recommendations for measures and future steps for the community to become more accessible and establish open data practices. We thus emphasize the potential of making research data available to enhance productivity, transparency, and reproducibility in the CER community. KW - open data, open science, datasets, reusing data, computing education, programming process data, secondary research, educational data mining Y1 - 2024 U6 - https://doi.org/10.1145/3598579.3689378 SP - 31 EP - 60 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Kiesler, Natalie A1 - Schiffner, Daniel T1 - What is a Good API? A Survey on the Use and Design of Application Programming Interfaces T2 - Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering Y1 - 2024 SN - 9783031515712 U6 - https://doi.org/10.1007/978-3-031-51572-9_4 SN - 1867-8211 SP - 45 EP - 55 PB - Springer Nature Switzerland CY - Cham ER - TY - CHAP A1 - Azaiz, Imen A1 - Kiesler, Natalie A1 - Strickroth, Sven T1 - Feedback-Generation for Programming Exercises With GPT-4 T2 - Proceedings of the 2024 on Innovation and Technology in Computer Science Education Vol. 1 N2 - Ever since Large Language Models (LLMs) and related applications have become broadly available, several studies investigated their potential for assisting educators and supporting students in higher education. LLMs such as Codex, GPT-3.5, and GPT 4 have shown promising results in the context of large programming courses, where students can benefit from feedback and hints if provided timely and at scale. This paper explores the quality of GPT-4 Turbo's generated output for prompts containing both the programming task specification and a student's submission as input. Two assignments from an introductory programming course were selected, and GPT-4 was asked to generate feedback for 55 randomly chosen, authentic student programming submissions. The output was qualitatively analyzed regarding correctness, personalization, fault localization, and other features identified in the material. Compared to prior work and analyses of GPT-3.5, GPT-4 Turbo shows notable improvements. For example, the output is more structured and consistent. GPT-4 Turbo can also accurately identify invalid casing in student programs' output. In some cases, the feedback also includes the output of the student program. At the same time, inconsistent feedback was noted such as stating that the submission is correct but an error needs to be fixed. The present work increases our understanding of LLMs' potential, limitations, and how to integrate them into e-assessment systems, pedagogical scenarios, and instructing students who are using applications based on GPT-4. Y1 - 2024 U6 - https://doi.org/10.1145/3649217.3653594 SP - 31 EP - 37 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - McCauley, Renée A1 - Sabin, Mihaela A1 - Kumar, Amruth N. A1 - Kiesler, Natalie A1 - MacKellar, Bonnie A1 - Raj, Rajendra K. A1 - Impagliazzo, John T1 - Using Vignettes to Elicit Students' Understanding of Dispositions in Computing Education T2 - 2023 IEEE Frontiers in Education Conference (FIE) N2 - Vignettes are short stories along with a set of questions that engage the reader to comment on the story. Vignettes have been used in professional academic programs (e.g., teacher preparation and medical education), for professional development in various fields (e.g., teaching ethics in psychology and medicine), and in various research fields for data collection. In this work, vignettes are used to elicit students’ understanding of dispositions in computing education. Professional dispositions enable behaviors that are valued in the workplace, such as adaptability or self-directedness. They are often explicitly stated in computing job postings. While the relevance of dispositions is widely recognized in the workplace, only recently have curricular guidelines for computing programs recognized professional dispositions as an integral part of competencies and as complementary to knowledge and skills. There is scarce literature on the use of vignettes in teaching undergraduate computing, or on how best to foster dispositions in students. In this project, four faculty from four diverse institutions in the U.S., along with three consulting experts, have collaborated to design and evaluate the use of vignettes in the classroom. This paper documents researchers’ efforts to gain insights into students’ perceptions of dispositions through the use of vignettes. Such insights may guide educators to identify pedagogical strategies for fostering dispositions among students. This paper presents an iterative process for vignette design with continuous review by researchers and focus group members. The vignettes in this study use stories of situations which demonstrate the application of a disposition, drawn from various fields and walks of life to represent diverse groups and experiences. Students are presented with the vignette story and asked to identify the disposition illustrated. To elicit students’ understanding of dispositions in terms of their personal behaviors, students are asked to describe a situation in which they have experienced the disposition. Lessons learned in the design and use of vignettes are discussed. KW - Dispositions, vignettes, professional practice Y1 - 2023 U6 - https://doi.org/10.1109/FIE58773.2023.10342915 SP - 1 EP - 5 PB - IEEE ER - TY - CHAP A1 - Kiesler, Natalie A1 - Impagliazzo, John T1 - Evaluating Diversity, Equity, Inclusion, and Accessibility in Accreditation T2 - 2023 International Symposium on Accreditation of Engineering and Computing Education (ICACIT) N2 - The advantages of the current digital evolution foster the responsibility of having technical advancements, applications, and tools reflect the societal needs regarding diversity, equality, inclusion, and accessibility (DEIA) within disciplines and organizations. These four elements should become part of all accreditation criteria and processes for engineering, computing, and other related programs, where accreditation bodies, higher education institutions, and educators become ready to implement these elements in any new setting. This work continues the discussion of an earlier paper that addresses the meaning of three of these four concepts in the context of accreditation. It also emphasizes the potential benefits for technical professions, future engineers, and the ICACIT accrediting agency. Hence, it is essential not only to integrate diversity, equity, inclusion, and accessibility in higher education study programs and global accreditation criteria but also to seek both quantitative and qualitative methods to evaluate these DEIA elements to ensure all students receive the best possible university education. Y1 - 2023 U6 - https://doi.org/10.1109/ICACIT59946.2023.10403684 SP - 1 EP - 5 PB - IEEE ER - TY - CHAP A1 - Kiesler, Natalie A1 - Impagliazzo, John T1 - Industry's Expectations of Graduate Dispositions T2 - 2023 IEEE Frontiers in Education Conference (FIE) N2 - This work represents a work-in-progress study on how dispositions are essential to the workplace. Recent computing curricular reports have heralded the importance of moving from knowledge-based to competency-based learning. However, dispositions, as with skills and knowledge, are a crucial component of competency. Hence, this work reports on a study conducted by the authors to ascertain the relationship between job advertisements and the dispositions expected in the workplace. The results show a remarkable degree of correlation between the two. At the same time, some dispositions are more explicitly expected than others. Hence, computing and engineering educators should increase their efforts to foster dispositions in their curricula to develop competent graduates ready to succeed in the workplace. At the same time, more research is required on the industry's perspective and understanding dispositions. Y1 - 2023 U6 - https://doi.org/10.1109/FIE58773.2023.10343406 SP - 1 EP - 5 PB - IEEE ER - TY - CHAP A1 - Kiesler, Natalie A1 - Kumar, Amruth N. A1 - MacKellar, Bonnie K. A1 - McCauley, Renée A1 - Sabin, Mihaela A1 - Impagliazzo, John T1 - Students' Perceptions of Behaviors Associated with Professional Dispositions in Computing Education T2 - Proceedings of the 2024 on Innovation and Technology in Computer Science Education Vol. 1 N2 - Dispositions, skills, and knowledge form the three components of competency-based education. Moreover, dispositions are considered crucial for students to succeed in the workplace. Few studies investigate how dispositions manifest in the form of observable behaviors, which causes challenges for both students and educators. Computing students, for example, may not understand what is expected of them, and how to achieve dispositions. This paper presents the results of a qualitative, multi-institutional study on students’ understanding of the dispositions adaptable, persistent, self-directed, meticulous, and professional. Perceptions were gathered by asking for exemplary situations of students applying each of the five dispositions in the context of assignments within computing courses. Students who indicated they did not apply the disposition were asked to describe the hindering circumstances. The data was evaluated by using Mayring’s content analysis technique, resulting in the development of deductive-inductive categories of observable behaviors reflecting the student’s perspective. For meticulous and professional, new categories representing observable behaviors were developed. For adaptable, persistent, and self-directed, the authors confirmed and extended prior work. Moreover, factors hindering students in applying the investigated dispositions are identified. The resulting categories with observable student behaviors are an important step toward the operationalization of competency-based learning outcomes including dispositions. A common understanding of dispositions will also help with the design of new forms of instruction and measures to foster the application of dispositions in the context of computing education. KW - Dispositions, meticulous, professional, competency-based educa- tion, observable learning outcomes, CC2020 Computing Curricula, qualitative analysis, student perspective Y1 - 2024 U6 - https://doi.org/10.1145/3649217.3653566 SP - 353 EP - 359 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Lohr, Dominic A1 - Kiesler, Natalie A1 - Keuning, Hieke A1 - Jeuring, Johan T1 - "Let Them Try to Figure It Out First" - Reasons Why Experts (Do Not) Provide Feedback to Novice Programmers T2 - Proceedings of the 2024 on Innovation and Technology in Computer Science Education Vol. 1 N2 - A recent ITiCSE working group investigated when and how experts give feedback and hints at steps novice programmers take when solving programming problems. Based on the feedback literature and an analysis of expert feedback on steps, the working group designed guidelines for when and how to give feedback. The feed-back provided by educators using these guidelines on a number of sequences of student steps varied a lot. In this paper, we try to answer the question of why educators give feedback at particular steps to novice learners of programming. We prepared six authentic sequences of student steps when solving an introductory programming task. The preprocessed sequences were used in a survey to gather information about when and why an expert would give feedback. Respondents annotated each step from one sequence with if and why they would give feedback at that step. Our survey received 47 responses. We qualitatively analyzed the responses, resulting in a coding scheme consisting of 19 different reasons for why experts intervene (or not) when novice learners work on introductory programming tasks. We found a considerable variety of reasons experts give for when and how to help students with feedback and hints. Also, sometimes one expert uses a reason at a step to explain why they do intervene, and another expert uses the same reason at the step to not intervene. The categories of experts’ feedback indicators will pave the way for several future studies and applications, including learning systems trying to resemble expert feedback strategies. KW - Learning programming, expert feedback, feedback rationale, feed- back guidelines, novice programmers Y1 - 2024 U6 - https://doi.org/10.1145/3649217.3653530 SP - 38 EP - 44 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Kiesler, Natalie A1 - Scholz, Ingo A1 - Albrecht, Jens A1 - Stappert, Friedhelm A1 - Wienkop, Uwe T1 - Novice Learners of Programming and Generative AI - Prior Knowledge Matters T2 - Proceedings of the 24th Koli Calling International Conference on Computing Education Research N2 - With the broad availability of Generative AI (GenAI), introductory programming education is starting to change. At Nuremberg Tech, we observed the doubling of failure rates to approximately 50% in the first semester course “Procedural Programming” across students of all study programs. Due to these exam results in winter 2023/24, we conducted a pilot study to gather students’ use of GenAI tools, their exam results, and prior programming education and experience. The results imply significant differences of students’ use of GenAI tools depending on their prior programming education. We will therefore extend the investigation in winter term 2024/25. Y1 - 2024 U6 - https://doi.org/10.1145/3699538.3699580 SP - 1 EP - 2 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Keuning, Hieke A1 - Luxton-Reilly, Andrew A1 - Ott, Claudia A1 - Petersen, Andrew A1 - Kiesler, Natalie T1 - Goodbye Hello World - Research Questions for a Future CS1 Curriculum T2 - Proceedings of the 24th Koli Calling International Conference on Computing Education Research N2 - Generative AI (GenAI) is currently capable of generating correct code for introductory level programming problems, and its performance is improving. We believe that this capability can be leveraged to improve student motivation, broaden students’ understanding of software development, and engage them in more authentic learning. We defined a set of assumptions about GenAI’s future capabilities (e.g., the ability to generate small pieces of code and to compose these pieces of code via user prompts) and engaged in a backcasting exercise to identify what else is needed to develop a CS1 course that places GenAI in a central role. Undertaking this thought experiment immediately revealed that aspects of the software development process usually reserved for later in the curriculum, such as requirements elicitation and design, could be introduced earlier in the process. With GenAI tools bearing the load of generating correct code snippets, students could focus on higher-level software design and construction skills and practice them in an authentic environment. Our thought experiment identified a set of questions that need to be addressed for such a course to actually exist, including questions about student preparation, and the ability of students to decompose problems effectively and to resolve problems that arise when integrating pieces of code. We also identified questions related to the design of a GenAI centered course, such as the impact on student motivation of using GenAI instead of engaging directly with code, the extent to which social learning theories apply to interactions with GenAI, and how existing pedagogies can integrate GenAI tools. Y1 - 2024 U6 - https://doi.org/10.1145/3699538.3699591 SP - 1 EP - 2 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Alshaigy, Bedour A1 - Grande, Virginia A1 - Kiesler, Natalie A1 - Settle, Amber T1 - How Do You Solve A Problem Like Recruitment? On The Hiring and Retention of Computing Academics T2 - Proceedings of the 2024 on ACM Virtual Global Computing Education Conference Vol. 1 N2 - This paper critically examines persistent inequities in existing computing faculty hiring and retention practices, which gravely impact computing educators from marginalized groups. Throughout these processes, applicants fight against multiple systemic barriers, including but not limited to, biased job ads and discriminatory interview practices. The increasing use of generative AI tools to aid in tasks connected to the hiring process, such as writing recommendation letters, exacerbates these biases. The inequities persist despite global initiatives and legal mandates and serve as a direct contradiction to widespread institutional commitments to diversity and inclusion. By building on literature and the lived experiences of the SIGCSE community represented in a recent Technical Symposium session, we raise concerns about the different stages of this process, highlighting the importance of clear expectations and adequate support. The paper concludes with a call to align hiring practices with inclusive institutional values, requiring the academic community to reflect on and revise hiring policies for a more equitable future. It is of paramount importance to address the role of these practices in the erosion of marginalized communities from the computing education community, a marginalization that occurs in many different contexts and negatively impacts everyone involved. KW - CS academics, recruitment, retention Y1 - 2024 U6 - https://doi.org/10.1145/3649165.3703622 SP - 263 EP - 266 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Kiesler, Natalie A1 - Schiffner, Daniel T1 - Conferences are Exclusive by Nature T2 - Proceedings of the 2024 on RESPECT Annual Conference N2 - This paper presents a perspective on equity by examining the inherent exclusivity of academic conferences within the context of computing education research. Our perspective delves into unequal opportunities faced by (young) researchers, which are rooted in the challenges of limited mobility, finances, and, for example, family obligations preventing equitable participation. By identifying and interrogating these barriers, the paper advocates for a reevaluation of conference structures to foster inclusivity and address the systemic issues that disproportionately impact certain demographics. Therefore, this paper contributes to the ongoing discourse on equity within academic spaces, urging a transformative approach to conference organization. KW - equity, diversity, conferences, resources, young researchers, mobility, finances, inclusion, sustainability Y1 - 2024 U6 - https://doi.org/10.1145/3653666.3656077 SP - 288 EP - 292 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Kiesler, Natalie A1 - Lohr, Dominic A1 - Keuning, Hieke T1 - Exploring the Potential of Large Language Models to Generate Formative Programming Feedback T2 - 2023 IEEE Frontiers in Education Conference (FIE) N2 - Ever since the emergence of large language models (LLMs) and related applications, such as ChatGPT, its performance and error analysis for programming tasks have been subject to research. In this work-in-progress paper, we explore the potential of such LLMs for computing educators and learners, as we analyze the feedback it generates to a given input containing program code. In particular, we aim at (1) exploring how an LLM like ChatGPT responds to students seeking help with their introductory programming tasks, and (2) identifying feedback types in its responses. To achieve these goals, we used students' programming sequences from a dataset gathered within a CS1 course as input for ChatGPT along with questions required to elicit feedback and correct solutions. The results show that ChatGPT performs reasonably well for some of the introductory programming tasks and student errors, which means that students can potentially benefit. However, educators should provide guidance on how to use the provided feedback, as it can contain misleading information for novices. Y1 - 2023 U6 - https://doi.org/10.1109/FIE58773.2023.10343457 PB - IEEE ER - TY - CHAP A1 - Scarcella, Diego A1 - Schneider, Jan A1 - Kiesler, Natalie A1 - Schiffner, Daniel T1 - Mathkinetics: Solving Arithmetics While Running out of Breath T2 - Proceedings of the 16th International Conference on Computer Supported Education N2 - To benefit from most of the current digital educational technologies, learners are required to sit down and look closely at a computer monitor or smart device screen for hours, which can have side effects on learners’ health and lifestyle. As an attempt to address this, we developed MathKinetics, an application designed to support the practice of cognitive skills such as arithmetic while engaging in physical activity by integrating the principles of Multimodal Learning, Life Kinetik, and Gamification. MathKinetics is a variant of an endless running game where users control an avatar through their body posture and dodge obstacles. At the same time, they pick up arithmetic problems whose answers need to be verbalized. In this paper, we present an exploratory evaluation of MathKinetics and its user experience. We conducted user tests with 20 participants. Results from our tests indicate that MathKinetics is a fun way to practice arithmetic skills and train executive cognitive functions such as task switching. KW - Game-Based Learning, Multimodal Learning, Natural Interaction Y1 - 2024 SN - 978-989-758-697-2 U6 - https://doi.org/10.5220/0012536900003693 SN - 2184-5026 SP - 250 EP - 256 PB - SCITEPRESS - Science and Technology Publications ER - TY - CHAP A1 - Kiesler, Natalie A1 - Röpke, René A1 - Schiffner, Daniel A1 - Schulz, Sandra A1 - Strickroth, Sven A1 - Ehlenz, Matthias A1 - Heinemann, Birte A1 - Wilhelm-Weidner, Arno ED - Schulz, Sandra ED - Kiesler, Natalie T1 - Towards Open Science at the DELFI Conference T2 - 22. Fachtagung Bildungstechnologien (DELFI) N2 - Despite the increasing awareness of Open Science within the educational technology community, conferences, such as DELFI, do not yet foster the publication of research data including software. To address this, we conducted a survey eliciting the community’s needs, perspectives, and publication preferences. The analysis of 24 valid responses reveals a variety of research data formats used, and several uncertainties, e. g., regarding data ownership. Associated barriers comprise legal concerns and lacking resources to publish data. Nonetheless, researchers seem open for new publication formats. Moreover, we analyzed author’s intentions to publish data related to their DELFI submissions in 2023 (n=66). Many researchers assume not to have data to share (n=28), or no intention to publish data in the future (n=16). Overall, the results imply a lack of awareness and recognition of data publications, so that further efforts and incentives are required to move toward Open Science practices in the DELFI community. KW - Open Science , Open Data , Data publication , FAIR , DELFI , Educational Technology Y1 - 2024 U6 - https://doi.org/10.18420/delfi2024_22 SP - 251 EP - 265 ER - TY - CHAP A1 - Scholl, Andreas A1 - Schiffner, Daniel A1 - Kiesler, Natalie ED - Schulz, Sandra ED - Kiesler, Natalie T1 - Analyzing Chat Protocols of Novice Programmers Solving Introductory Programming Tasks with ChatGPT T2 - 22. Fachtagung Bildungstechnologien (DELFI) N2 - Large Language Models (LLMs) have taken the world by storm, and students are assumed to use related tools at a great scale. In this research paper we aim to gain an understanding of how introductory programming students chat with LLMs and related tools, e.g., ChatGPT-3.5. To address this goal, computing students at a large German university were motivated to solve programming exercises with the assistance of ChatGPT as part of their weekly introductory course exercises. Then students (n=213) submitted their chat protocols (with 2335 prompts in sum) as data basis for this analysis. The data was analyzed w.r.t. the prompts, frequencies, the chats’ progress, contents, and other use pattern, which revealed a great variety of interactions, both potentially supportive and concerning. Learning about students’ interactions with ChatGPT will help inform and align teaching practices and instructions for future introductory programming courses in higher education. KW - ChatGPT-3.5, large language models, students, interaction pattern, application, chat protocols, introductory programming, higher education Y1 - 2024 U6 - https://doi.org/10.18420/delfi2024_05 SN - 2944-7682 SP - 63 EP - 79 PB - Gesellschaft für Informatik e.V. CY - Bonn ER - TY - CHAP A1 - Schiffner, Daniel A1 - Kiesler, Natalie ED - Kiesler, Natalie ED - Schulz, Sandra T1 - Open Science in den Bildungstechnologien: Unterstützung bei der Veröffentlichung von Forschungsdaten T2 - Proceedings of the DELFI 2024 Workshops N2 - Der zweite Workshop des Arbeitskreises Open Science zielte darauf ab, gemeinsam mit der Fachcommunity mögliche Verfahren zur Publikation von Forschungsdaten (inklusive Software) aus Sicht der Autorinnen und Autoren zu erörtern. Dazu gehörte sowohl die Entwicklung realistischer und nachvollziehbarer Anforderungen an die Publikation von Forschungsdaten inklusive Software, als auch die bisherigen Herausforderungen für Autoren(-teams). Darüber hinaus wurden neue, mögliche Publikationsformate und Konferenztracks diskutiert. Der Workshop basierte stark auf der Interaktion mit den Teilnehmenden, um deren Perspektive aufgreifen zu können. Mit diesem Beitrag werden die wichtigsten Ergebnisse des halbtägigen Workshops zusammengefasst, und so unter anderem mit dem Leitungsgremium, zukünftigen DELFI-Verantwortlichen sowie dem Projekt Nationale Forschungsdateninfrastruktur für und mit der Informatik (NFDIxCS) geteilt. KW - Open Science, FAIR Prinzipien, Software, Forschungsdaten, Autor*innen Unterstützung, Publikationsformate, NFDIxCS Y1 - 2024 U6 - https://doi.org/10.18420/delfi2024-ws-11 SP - 77 EP - 85 PB - Gesellschaft für Informatik e.V. CY - Bonn ER - TY - CHAP A1 - MacNeil, Stephen A1 - Leinonen, Juho A1 - Denny, Paul A1 - Kiesler, Natalie A1 - Hellas, Arto A1 - Prather, James A1 - Becker, Brett A. A1 - Wermelinger, Michel A1 - Reid, Karen T1 - Discussing the Changing Landscape of Generative AI in Computing Education T2 - Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2 N2 - In a previous Birds of a Feather discussion, we delved into the nascent applications of generative AI, contemplating its potential and speculating on future trajectories. Since then, the landscape has continued to evolve revealing the capabilities and limitations of these models. Despite this progress, the computing education research community still faces uncertainty around pivotal aspects such as (1) academic integrity and assessments, (2) curricular adaptations, (3) pedagogical strategies, and (4) the competencies students require to instill responsible use of these tools. The goal of this Birds of a Feather discussion is to unravel these pressing and persistent issues with computing educators and researchers, fostering a collaborative exploration of strategies to navigate the educational implications of advancing generative AI technologies. Aligned with this goal of building an inclusive learning community, our BoF is led by globally distributed leaders to facilitate multiple coordinated discussions that can lead to a broader conversation about the role of LLMs in CS education. KW - academic integrity; assessment; computing education; curriculum; large language models; pedagogy Y1 - 2024 SN - 979-8-4007-0424-6 U6 - https://doi.org/10.1145/3626253.3635369 PB - ACM CY - New York, NY, USA ER -