Refine
Document Type
- Conference Proceeding (6)
- Article (2)
Language
- English (8)
Keywords
- VSVR (2)
- virtual studio (2)
- Poster (1)
- Virtual (TV) Studio (1)
- camera-tracking (1)
- context awareness (1)
- context query language (1)
- development methodology (1)
- internet of things (1)
- meta model (1)
Department/institution
In this paper we describe a prototypical system for live musical performance in a virtual studio environment. The performer stands in front of the studio camera and interacts with an infrared-laser-based multi-touch device. The final TV image shows the performer interacting with a virtual screen which is augmented in front of herself. To overcome the problem of the performer not seeing this virtual screen in reality, we use a special hexagonal grid to facilitate the performer's awareness of this novel Theremin-like virtual musical instrument.
This paper presents a tracking of parts of a human body in a virtual TV studio environment. The tracking is based on a depth camera and a HD studio camera and aims at a realistic interaction between the actor and the computer generated environment. Stereo calibration methods are used to match corresponding pixels of both cameras (HD color and depth image). Hence the images were rectified and column aligned. The disparity is used to correct the depth image pixel by pixel. This image registration results in row and column aligned images where ghost regions are in the depth image resulting from occlusion. Both images are used to generate foreground masks with chroma and depth keying. The color image is taken for skin color segmentation to determine and distinguish the actor’s hands and face. In the depth image the flesh colored regions were used to determine their spatial position. The extracted positions were augmented by virtual objects. The scene is rendered correctly with virtual camera parameters which were calculated from the camera calibration parameters. Generated computer graphics with alpha value are combined with the HD color images. This compositing shows interaction with augmented objects for verification. The additional depth information results in changing the size of objects next to the hands when the actor moves around.
Building context-aware applications is an already widely researched topic. It is our belief that context awareness has the potential to supplement the Internet of Things, when a suitable methodology including supporting tools will ease the development of context-aware applications. We believe that a meta-model based approach can be key to achieving this goal. In this paper, we present our meta-model based methodology, which allows us to define and build application-specific context models and the integration of sensor data without any programming. We describe how that methodology is applied with the implementation of a relatively simple context-aware COVID-safe navigation app. The outcome showed that programmers with no experience in context-awareness were able to understand the concepts easily and were able to effectively use it after receiving a short training. Therefore, context-awareness is able to be implemented within a short amount of time. We conclude that this can also be the case for the development of other context-aware applications, which have the same context-awareness characteristics. We have also identified further optimization potential, which we will discuss at the conclusion of this article.