Looking Inside - Mobile Screen Recordings as a Privacy Friendly Long-Term Data Source to Analyze User Behavior

  • Mobile devices are ubiquitous in many societies and shape the way we interact with technology and each other. Research on how we use and perceive technology is essential to understand its impact. This work advances how we can follow user behavior on mobile devices. We combine the strength of two common data sources for tracking on mobile devices, log files, and screen recordings. Log files are suitable for long-term and privacy-friendly analyzation but provide rather general data (e.g. system log files) unless one has access to the source code of the applications or operating systems. Screen recordings are usually used for short-termed analysis (e.g. usability tests) because the analysis is time-consuming, but they provide all activities on the screen in high detail regardless of which application or operating system. This thesis combines both data sources and presents an approach to automatically generate log files from mobile screen recordings. The approach utilizes methods ofMobile devices are ubiquitous in many societies and shape the way we interact with technology and each other. Research on how we use and perceive technology is essential to understand its impact. This work advances how we can follow user behavior on mobile devices. We combine the strength of two common data sources for tracking on mobile devices, log files, and screen recordings. Log files are suitable for long-term and privacy-friendly analyzation but provide rather general data (e.g. system log files) unless one has access to the source code of the applications or operating systems. Screen recordings are usually used for short-termed analysis (e.g. usability tests) because the analysis is time-consuming, but they provide all activities on the screen in high detail regardless of which application or operating system. This thesis combines both data sources and presents an approach to automatically generate log files from mobile screen recordings. The approach utilizes methods of computer vision and machine learning to automatically process screen recordings and extend their use. Screen recordings reveal virtually everything a user does with a device, making privacy important, especially in user studies. We present a privacy concept and implementation and show how the risk of exposing private data can be reduced, by processing all recordings locally on the mobile devices and anonymizing the resulting log files. In order to apply the developed method in practice, we carry out a study in the context of education and show how log files of screen recordings can complement and extend existing research in learning analytics. This thesis opens up novel perspectives on how we can look at human-computer interaction with mobile devices. We show how to generate long-term log data with high detail and accuracy from mobile screen recordings, in a privacy-friendly way, locally on mobile devices.show moreshow less

Export metadata

Metadaten
Author:Philipp KrieterORCiD
open access:Bronze - frei zugänglich aber ohne Lizenzhinweis, d.h. keine Weitergabe
Fachbereich/Einrichtung:Hochschule Düsseldorf / Fachbereich - Medien
Document Type:Doctoral Thesis
Year of Completion:2020
Referee:Andreas BreiterORCiD, Barry BrownORCiD
Language of Publication:English
Publisher:Bremen
Place of publication:Universität Bremen
URN:urn:nbn:de:gbv:46-elib43189
DOI:https://doi.org/10.26092/elib/103
Tag:learning analytics; log files; mobile screen recordings; privacy; user behavior
Dewey Decimal Classification:0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 000 Informatik, Informationswissenschaft, allgemeine Werke
Licence (German):Es gilt das deutsche Urheberrecht
Release Date:2025/04/03
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.