Analyzing Chat Protocols of Novice Programmers Solving Introductory Programming Tasks with ChatGPT

  • Large Language Models (LLMs) have taken the world by storm, and students are assumed to use related tools at a great scale. In this research paper we aim to gain an understanding of how introductory programming students chat with LLMs and related tools, e.g., ChatGPT-3.5. To address this goal, computing students at a large German university were motivated to solve programming exercises with the assistance of ChatGPT as part of their weekly introductory course exercises. Then students (n=213) submitted their chat protocols (with 2335 prompts in sum) as data basis for this analysis. The data was analyzed w.r.t. the prompts, frequencies, the chats’ progress, contents, and other use pattern, which revealed a great variety of interactions, both potentially supportive and concerning. Learning about students’ interactions with ChatGPT will help inform and align teaching practices and instructions for future introductory programming courses in higher education.

Export metadata

Additional Services

Search Google Scholar
Metadaten
Author:Andreas Scholl, Daniel Schiffner, Natalie Kiesler
DOI:https://doi.org/10.18420/delfi2024_05
ISSN:2944-7682
Parent Title (English):22. Fachtagung Bildungstechnologien (DELFI)
Publisher:Gesellschaft für Informatik e.V.
Place of publication:Bonn
Editor:Sandra Schulz, Natalie Kiesler
Document Type:conference proceeding (article)
Language:English
Reviewed:Begutachtet/Reviewed
Release Date:2024/12/17
Tag:ChatGPT-3.5, large language models, students, interaction pattern, application, chat protocols, introductory programming, higher education
Pagenumber:17
First Page:63
Last Page:79
institutes:Fakultät Informatik
Research Themes:Digitalisierung & Künstliche Intelligenz
Licence (German):Creative Commons - CC BY-SA - Namensnennung - Weitergabe unter gleichen Bedingungen 4.0 International
Verstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.