From Promise to Proof: Assessing Replicability and Reproducibility in Interaction-Based Emotion Recognition Research
- Independent verification of published results is crucial for ensuring research quality and promoting further advancements. However, reproducibility and replicability in data-focused and machine learning research are growing concerns across disciplines. Our research aims to improve the reproducibility and replicability of interaction-based emotion recognition research. For this, we evaluate current publications based on the ability to reproduce or replicate their results. We survey these practices by trying to verify the results from 14 publications published between 2014 and 2024. For those 14 publications, we examined their code sharing practices. Eight out of the 14 publications shared the code used for the analysis, while six did not. For those who shared their code, we tried to reproduce their results. We were able to fully reproduce only three out of eight publications. Furthermore, we could at least partially reproduce the approaches of four further publications. For the six publications without shared code, we attempted toIndependent verification of published results is crucial for ensuring research quality and promoting further advancements. However, reproducibility and replicability in data-focused and machine learning research are growing concerns across disciplines. Our research aims to improve the reproducibility and replicability of interaction-based emotion recognition research. For this, we evaluate current publications based on the ability to reproduce or replicate their results. We survey these practices by trying to verify the results from 14 publications published between 2014 and 2024. For those 14 publications, we examined their code sharing practices. Eight out of the 14 publications shared the code used for the analysis, while six did not. For those who shared their code, we tried to reproduce their results. We were able to fully reproduce only three out of eight publications. Furthermore, we could at least partially reproduce the approaches of four further publications. For the six publications without shared code, we attempted to replicate their analyses but failed for all of them. Overall, we could verify at least to some extent the results of seven of 14 publications. This shows an overall low verifiability of existing research in that area, which hinders the reutilization and comparability of research results. In the future, it is necessary to encourage reproducibility in publications. We provide some recommendations to increase reproducibility and replicability in future publications.…
