AI-based User Emotion Recognition from Interaction Data : Challenges and Guidelines for Training Data Creation

  • Artificial Intelligence (AI) is a rising topic in the field of emotion recognition, e.g., from facial expressions. However, existing methods often require to be performed in staged set ups and are obtrusive by gathering additional data. Especially, collecting video data includes a high data protection risk. Our approach is to provide an unobtrusive emotion recognition tool based on Keystroke, Mouse and Touchscreen (KMT) data. Recently, we published a data set for emotion recognition from keystroke and mouse interaction data. In this paper, we present the challenges we faced during the creation of the data set. This covers collecting User Interface (UI) data as well as emotional ground truth data. For each of seven mentioned challenges, we provide our solutions as well as guidelines for other researchers to prevent them. The challenges include possible issues with recorded data as well as issues of automated facial coding engines. We provide a possible approach for manual facial coding and describe aspects attention should be paid to.Artificial Intelligence (AI) is a rising topic in the field of emotion recognition, e.g., from facial expressions. However, existing methods often require to be performed in staged set ups and are obtrusive by gathering additional data. Especially, collecting video data includes a high data protection risk. Our approach is to provide an unobtrusive emotion recognition tool based on Keystroke, Mouse and Touchscreen (KMT) data. Recently, we published a data set for emotion recognition from keystroke and mouse interaction data. In this paper, we present the challenges we faced during the creation of the data set. This covers collecting User Interface (UI) data as well as emotional ground truth data. For each of seven mentioned challenges, we provide our solutions as well as guidelines for other researchers to prevent them. The challenges include possible issues with recorded data as well as issues of automated facial coding engines. We provide a possible approach for manual facial coding and describe aspects attention should be paid to. Furthermore, we indicate issues when using different software tools to collect the data. The paper aims to help other researchers by providing insights and a guideline for the creation of the data set. We make these insights available for other researchers who want to create similar data sets or who want to expand ours. By sharing our insights, we aim to improve the reproducibility of AI training data creation and AI-based emotion recognition from user interaction data.show moreshow less

Export metadata

Additional Services

Search Google Scholar
Metadaten
Author:Carina BieberORCiD, Patrick HarmsORCiD, Dominick Leppich, Katrin Proschek
DOI:https://doi.org/10.1145/3652620.3686245
Parent Title (English):MODELS Companion ´24
Document Type:conference proceeding (article)
Language:English
Date of first Publication:2024/10/31
Reviewed:Begutachtet/Reviewed
Release Date:2025/02/05
Tag:data set, artificial intelligence, emotion recognition, user interaction
Pagenumber:242 - 252
Konferenzangabe:27th International Conferene on Model Driven Engineering Language and Systems
institutes:Fakultät für Elektrotechnik Feinwerktechnik Informationstechnik
Ohm User Experience Center (Ohm-UX)
Research Themes:Digitalisierung & Künstliche Intelligenz
Licence (German):Creative Commons - CC BY - Namensnennung 4.0 International
Verstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.