Refine
Document Type
Language
- English (4)
Has Fulltext
- no (4)
Reviewed
Keywords
Institute
Is part of the Bibliography
- yes (4)
Artificial Intelligence (AI) is a rising topic in the field of emotion recognition, e.g., from facial expressions. However, existing methods often require to be performed in staged set ups and are obtrusive by gathering additional data. Especially, collecting video data includes a high data protection risk. Our approach is to provide an unobtrusive emotion recognition tool based on Keystroke, Mouse and Touchscreen (KMT) data. Recently, we published a data set for emotion recognition from keystroke and mouse interaction data. In this paper, we present the challenges we faced during the creation of the data set. This covers collecting User Interface (UI) data as well as emotional ground truth data. For each of seven mentioned challenges, we provide our solutions as well as guidelines for other researchers to prevent them. The challenges include possible issues with recorded data as well as issues of automated facial coding engines. We provide a possible approach for manual facial coding and describe aspects attention should be paid to. Furthermore, we indicate issues when using different software tools to collect the data. The paper aims to help other researchers by providing insights and a guideline for the creation of the data set. We make these insights available for other researchers who want to create similar data sets or who want to expand ours. By sharing our insights, we aim to improve the reproducibility of AI training data creation and AI-based emotion recognition from user interaction data.
User experience evaluation is becoming increasingly important, and so is emotion recognition. Recognizing users’ emotions based on their interactions alone would not be intrusive to users and can be implemented in many applications. This is still an area of active research and requires data containing both the user interactions and the corresponding emotions. Currently, there is no public dataset for emotion recognition from keystroke, mouse and touchscreen dynamics. We have created such a dataset for keyboard and mouse interactions through a dedicated user study and made it publicly available for other researchers. This paper examines our study design and the process of creating the dataset. We conducted the study using a test application for travel expense reports with 50 participants. We want to be able to detect predominantly negative emotions, so we added emotional triggers to our test application. However, further research is needed to determine the relationship between user interactions and emotions.
Data is a fundamental part of all data-driven and artificial intelligence-related research. Independent verification of published results is crucial for ensuring research quality and promoting further advancements. However, reproducibility in data-focused research is a growing concern across disciplines. Our research aims to improve the reproducibility of interaction-based emotion recognition research. For this, we evaluate current data sharing practices in this area. We survey these practices by examining 100 publications published from 2014 to 2024. We examined the characteristics of shared and not shared data, trends over time, and influences of authorship and publication type. We requested data from the publications’ authors for which the data was not shared as part of the publication. Afterwards, we evaluated the reasons we were given as to why the data could not be shared. Overall, we observed limited data sharing, with only 15 out of 100 publications providing their data, even after we sent requests for data to authors. We only received data for four publications after sending requests via email to the authors. Furthermore, the shared artifacts are often insufficient for full reproducibility. We found that, data was shared along a journal article more often than a conference publication. However, data sharing increased noticeably in recent years, particularly since 2021. Overall, the lack of data sharing hinders the reproducibility and comparability of the research results. In the future, it is necessary to encourage the data sharing along publications by introducing mandatory guidelines.