Refine
Year of publication
Document Type
- Conference Proceeding (55)
- Article (18)
- Part of a Book (12)
- Article trade magazine (2)
- Diploma Thesis (1)
- Doctoral Thesis (1)
Language
- English (78)
- German (10)
- Multiple languages (1)
Keywords
- VSVR (46)
- Virtual (TV) Studio (23)
- FHD (20)
- Sound Spatialization (8)
- Resource Management (6)
- audio rendering (5)
- human perception (5)
- virtual environments (5)
- virtual studio (5)
- Lehre (4)
Department/institution
- Creative Media Production and Entertainment Computing (89) (remove)
Live video broadcasting requires a multitude of professional expertise to enable multi-camera productions. Robotic systems allow the automation of common and repeated tracking shots. However, predefined camera shots do not allow quick adjustments when required due to unpredictable events. We introduce a modular automated robotic camera control and video switch system, based on fundamental cinematographic rules. The actors' positions are provided by a markerless tracking system. In addition, sound levels of actors' lavalier microphones are used to analyse the current scene. An expert system determines appropriate camera angles and decides when to switch from one camera to another. A test production was conducted to observe the developed prototype in a live broadcast scenario and served as a video-demonstration for an evaluation.
In this paper we describe a prototypical system for live musical performance in a virtual studio environment. The performer stands in front of the studio camera and interacts with an infrared-laser-based multi-touch device. The final TV image shows the performer interacting with a virtual screen which is augmented in front of herself. To overcome the problem of the performer not seeing this virtual screen in reality, we use a special hexagonal grid to facilitate the performer's awareness of this novel Theremin-like virtual musical instrument.
Given limited computational resources available for the rendering of spatial sound imagery, we seek to determine effective means for choosing whatcomponents of the rendering will provide the most audible differences in the results. Rather than begin with an analytic approach that attempts to predict audible differences on the basis of objective parameters, we chose to begin with subjective tests of how audibly different the rendering result may be heard to be when that result includes two types of sound obstruction: reflectors and occluders. Single-channel recordings of 90 short speech sounds were made in an anechoic chamber in the presence and absence of these two types of obstructions, and as the angle of those obstructions varied over a 90 degree range. These recordings were reproduced over a single loudspeaker in that anechoic chamber, and listeners were asked to rate how confident they were that the recording of each of these 90 stimuli included an obstruction. These confidence ratings can be used as an integral component in the evaluation function used to determine which reflectors and occluders are most important for rendering.
Level-of-detail is a concept well-known in computer graphics to reduce the number of rendered polygons. Depending on the distance to the subject (viewer), the objects’ representation is changed. A similar concept is the clustering of sound sources for sound spatialization. Clusters can be used to hierarchically organize mixels and to optimize the use of resources, by grouping multiple sources together into a single representative ource. Such a clustering process should minimize the error of position allocation of
elements, perceived as angle and distance, and also differences between velocity relative to the sink (i.e., Doppler shift). Objects with similar direction of motion and speed (relative to sink) in the same acoustic resolution cone and with similar distance to a sink can be grouped together.
Level-of-detail is a concept well-known in computer graphics to reduce the number of rendered polygons. Depending on the distance to the subject (viewer), the objects’ representation is changed. A similar concept is the clustering of sound sources for sound spatialization. Clusters can be used to hierarchically organize mixelsand to optimize the use of resources, by grouping multiple sources together into a single representative source. Such a clustering process should minimize the error of position allocation of elements, perceived as angle and distance, and also differences between velocity relative to the sink (i.e., Doppler shift). Objects with similar direction of motion and speed (relative to sink) in the same acoustic resolution cone and with similar distance to a sink can be grouped together.
The interpretation process of complex data sets makes the integration of effective interaction techniques crucial. Recent work in the field of human-computer interaction has shown that there is strong evidence that multimodal user interaction, i.e. the integration of various input modalities and interaction techniques into one comprehensive user interface, can improve human performance when interacting with complex data sets. However, it is still unclear which factors make these user interfaces superior to unimodal user interfaces. The contribution of this work is an analytical comparison of a multimodal and a unimodal user interface for a scientific visualization application. We show that multimodal user interaction with simultaneously integrated speech and gesture input improves user performance regarding efficiency and ease of use.
The article discusses the question of “How to convey the experience in a virtual environment to third parties?” and explains the different technical implementations which can be used for live streaming and recording of a mixed reality experience. The real-world applications of our approach include education, entertainment, e- sports, tutorials, and cinematic trailers, which can benefit from our research by finding a suitable solution for their needs. We explain and outline our Mixed Reality systems as well as discuss the experience of recorded demonstrations of different VR applications, including the need for calibrated camera lens parameters based on realtime encoder values.
We report about several experiments on applying mixed reality technology in the context of accessing collective memories from atomic bombs, Holocaust and Second World War. We discuss the impact of Virtual Reality, Augmented Virtuality and Augmented Reality for specific memorial locations. We show how to use a virtual studio for demonstrating an augmented reality application for a specific location in a remote session within a video conference. Augmented Virtuality is used to recreate the local environment, thus providing a context and helping the participants recollect emotions related to a certain place. This technique demonstrates the advantages of using virtual (VR) and augmented (AR) reality environments for rapid prototyping and pitching project ideas in a live remote setting.
High dynamic range environments maps based on still images or video streams are used for computer animation or interactive systems. The task of realistic light setup of scenes using captured environment maps might be eased as well as the visual quality improves. In this article, we discuss the light setting problem for virtual studio (tv) layout and system become more complex to handle this new feature of studio light capturing. The analysis of system requirements identifies the technical challenges.
Virtual (tv) studios gain much more acceptance through improvements in computer graphics and camera tracking. Still commercial studios cannot have full interaction between actors and virtual scene because actors data are not completely digital available as well as the feedback for actors is still not sufficient. Markerless full body tracking might revolutionize virtual studio technology as it allows better interaction between real and virtual world. This article reports about using a markerless actor tracking in a virtual studio with a tracking volume of nearly 40 cubic meter enabling up to three actors within the green box. The tracking is used for resolving the occlusion between virtual objects and actors so that the Tenderer can output automatically a mask for virtual objects in the foreground in case the actor is behind. It is also used for triggering functions scripted within the Tenderer engine, which are attached to virtual objects, starting any kind of action (e.g., animation). Last but not least the system is used for controlling avatars within the virtual set. All tracking and rendering is done within a studio frame rate of 50 Hz with about 3 frames delay. The markerless actor tracking within virtual studios is evaluated by experts using an interview approach. The statistical evaluation is based on a questionnaire.
The presentation of virtual environments in real time has always been a demanding task. Specially designed graphics hardware is necessary to deal with the large amounts of data these applications typically produce. For several years the chipsets that were used allowed only simple lighting models and fixed algorithms. But recent development has produced new graphics processing units (GPUs) that are much faster and more programmable than their predecessors. This paper presents an approach to take advantage of these new features. It uses a video texture as part of the lighting calculations for the passenger compartment of a virtual train and was run on the GPU of a recent PC graphics card. The task was to map the varying illumination of a filmed landscape onto the virtual objects and also onto another video texture (showing two passengers), thereby enhancing the realism of the scene.
In this paper we introduce a system for tracking persons walking or standing on a large planar surface and for using the acquired data to easily configure position based interactions for virtual studio productions. The tracking component of the system, radarTRACK, is based on a laser scanner device capable of delivering interaction points on a large configurable plane. By using the device on the floor it is possible to use the delivered data to detect feet positions and derive the position and orientation of one or more users in real time. The second component of the system, named OscCalibrator, allows for the easy creation of multidimensional linear mappings between input and output parameters and the routing of OSC messages within a single modular design environment. We demonstrate the use of our system to flexibly create position-based interactions in a virtual studio environment.
Der Programmier System Generator - PSG - des Fachgebiets Praktische Informatik in Darmstadt erzeugt aus einer Sprachdefinition eine sprachspezifische Programmierumgebung. Diese besteht u. a. aus einem Editor, welcher syntaktische und semantische Fehler von Programmfragmenten, die nicht vollständig sein müssen, erkennen kann. Dem Benutzer werden per Menü Fehlerkorrekturen angeboten. Neben der freien Texteingabe besteht die Möglichkeit, den Text nur mit Hilfe von Menüs zu verfeinern. Teil dieses Editors ist die Bezeichneranalyse. Sie dient als Hilfsmittel für den Benutzer, indem für jede Stelle eines Programmfragmentes die gültigen Bezeichner ausgegeben werden können. Die Kontextanalyse setzt die Berechnung auf den von der Bezeichneranalyse erzeugten Daten auf, um semantische Fehler zu erkennen. Die bis zu dieser Arbeit verwendete Bezeichneranalyse im PSG unterstützt nur einfache Sprachkonzepte (z. B. Fortran und Pascal). Die Gültigkeitskonzepte der Bezeichner von weiterentwickelten Sprachen (z. B. Modula-2, CHILL, Ada oder Pascal-XT) sind nicht vollständig modellierbar. Wir stellen ein neues Konzept zur Definition und Berechnung der Bezeichneranalyse vor, das alle uns bekannten Sprachen mit statischer Typbindung unterstützt. Hierfür haben wir die Sprache BIS - Bezeichneridentifikationssprache - definiert. Die Methode ist verwandt mid dem Zwischencode für geordnete Attributierte Grammatiken. Für jeden Knoten des Abstrakten Syntaxbaumes wird mit Hilfe von BIS ein Code für eine abstrakte Maschine, welche die Bezeichneranalyse durchführt, geschrieben. Im Gegensatz zu herkömlichen Methoden (verkettete Symboltabellen) wird für jeden Punkt innerhalb eines Programmes for der Anfrage durch den Benutzer oder der Kontextanalyse die Menge der gültigen Bezeichner berechnet. Die Kosten für eine Anfrage sind dadurch minimal. Diese abstrakte Maschine teilt sich in zwei unabhängige Maschinen auf, zum einen in die S-Maschine, die die speziellen Operationen der Bezeichneranalyse durchführt, und zum anderen in die G-Maschine, die den Datenfluss und die Auswertung steuert. Diese Aufteilung ermöglicht den Austausch der S-Maschine durch eine andere, welche neue Anwendungsgebiete erschliesst, z. B. die eines Praeprozessors. Die G-Maschine arbeitet inkrementell; es werden nur die Codeschablonen neu ausgewertet, deren geerbten Attribute sich geändert haben. Dazu müssen die Daten, die in einer Codeschablone hinein- und hinausfliessen, abgelegt werden. Dies ergibt bei grossen Programmfragmenten eine immense Rechenzeiteinsparung auf Kosten des Speicherplatzes. Die Funktionsweise wird an einer kleinen Beispielsprache demonstriert, die zu Pascal ähnlich ist. Diese besitzt Konstrukte zum Import und Export von Daten und Datentypen zwischen Programmfragmenten. Im Prototyp kann die inkrementelle Arbeitsweise abgeschaltet werden und ermöglicht einen guten Vergleich der Verfahren.
Die interaktive Echtzeit 3D-Visualisierung Mobilisierung und Homing von Blutstammzellen wurde konzipiert, um ein sehr komplexes medizinisches Wissen mit den Mitteln der 3-dimensionalen Visualisierung in Echtzeit und des Internets sowie der daraus resultierenden Interaktivität aufzubereiten. Dies musste auf einer Ebene geschehen, die es hinterher auch jedem Nicht-Mediziner erlaubt, die grundlegenden biologischen und medizinischen Sachverhalte nachzuvollziehen. Das Resultat: Eine informative und didaktische Anwendung, aus einer Mischung von interaktiven 3D-Stationen und erklärenden 3D-Animationen. Diskutiert werden die Methodik der Konzeptionsphase und die Interaktionstechniken.
Virtual set environments for broadcasting become more sophisticated as well as the visual quality improves. Realtime interaction and production-specific visualization implemented through plugin mechanism enhance the existing systems like the 3DK. This work presents the integration of the Intersense IS-900 SCT camera tracking and 3D interaction into the 3DK virtual studio environment. The main goal of this work is the design of a virtual studio environment for post productions, which includes video output as well as media streaming formats such as MPEG-4. The systems allows high quality offline rendering during post production and 3D interaction by the moderator during the recording.
Sound spatialization is a technology which puts sound into the three dimensional space, so that it has a perceivable direction and distance. Interactive means mutually or reciprocally active. Interaction is when one action (e.g., user moves mouse) has direct or immediate influence to other actions (e.g., processing by a computer: graphics change in size). Based on this definition an introduction to sound reproduction using DVD and virtual environments is given and illustrated by applications (e.g., virtual converts).
Virtual set environments for broadcasting become more sophisticated as well as the visual quality improves. Realtime interaction and production-specific visualization implemented through plugin mechanism enhance the existing systems like the virtual studio software 3DK. This work presents an algorithm which can dynamically manage textures of high resolution by prefetching them depending on their requirement in memory and map them on a procedural mesh in realtime. The main goal application of this work is the virtual representation of a flight over a landscape as part of weather reports in virtual studios and the interaction by the moderator.
Digital broadcasting enables interactive \sc tv, which presents new challenges for interactive content creation. Besides the technology for streaming and viewing, tools and systems are under development that extend traditional \sc tv studios with virtual set environments. This presentation reviews current technology and describes the requirements for such systems. Interoperability over the production, streaming, and viewer levels requires open interfaces. As the technology allow more interaction, it becomes inherent difficult to control the quality of the viewers experience