Refine
Year of publication
Document Type
- Conference Proceeding (55)
- Article (18)
- Part of a Book (12)
- Article trade magazine (2)
- Diploma Thesis (1)
- Doctoral Thesis (1)
Language
- English (78)
- German (10)
- Multiple languages (1)
Keywords
- VSVR (46)
- Virtual (TV) Studio (23)
- FHD (20)
- Sound Spatialization (8)
- Resource Management (6)
- audio rendering (5)
- human perception (5)
- virtual environments (5)
- virtual studio (5)
- Lehre (4)
- Mixed Reality (4)
- Virtual Reality (4)
- augmented reality (4)
- spatial media (4)
- Augmented Reality (3)
- Augmented Virtuality (3)
- groupware (3)
- spatialization (3)
- user interface design (3)
- Camera Tracking (2)
- Groupware (2)
- Haptik (2)
- Interaction (2)
- Interactive Media (2)
- M (2)
- Mensch-Maschine-Kommunikation (2)
- Mixed reality (2)
- Produkttest (2)
- TV (2)
- Tracking (2)
- Virtual Sets (2)
- Virtuelle Umgebungen (2)
- Visualization (2)
- actor tracking (2)
- architectural design (2)
- audio rendering, clustering (2)
- clustering, and human perception (2)
- design review (2)
- experimentation (2)
- first-order reflection (2)
- human factors (2)
- human-machine interfaces (2)
- interaction (2)
- interactive walkthrough (2)
- occluder (2)
- resource management (2)
- sound spatialization (2)
- spatial audio (2)
- spatial sound authoring (2)
- tactile feedback (2)
- tracking (2)
- video processing (2)
- virtual prototyping (2)
- virtual set environments (2)
- 3-D Interaction (1)
- 3D (1)
- 3D Interaktion (1)
- 3D audio (1)
- 3D interaction (1)
- 3D user interfaces (1)
- 3D-Echtzeitrendering (1)
- 3D-Rendering (1)
- Acoustic displays (1)
- Acoustic reflection (1)
- Animation (1)
- Auditory displays (1)
- Aural attributes (1)
- Automatische Bildauswahl (1)
- Avatar (1)
- Baustelle (1)
- Bildmischer (1)
- Blockchain (1)
- Blutstammzellen (1)
- Bot (1)
- CAI (1)
- CAT (1)
- CSCW (1)
- Cameras (1)
- Control systems (1)
- Cyborg (1)
- DVD (1)
- Delay effects (1)
- Delays (1)
- Distributed Virtual (1)
- Echtzeitanwendung (1)
- Education (1)
- Electronic mail (1)
- Engines (1)
- Februar (1)
- Filmregeln (1)
- GPU (1)
- Gesture recognition (1)
- HDR environment maps (1)
- HMD (1)
- Halbautomatische Roboterkamerasteuerung (1)
- HoloLens (1)
- Human (1)
- Human Computer Interaktion (1)
- Human-centered computing (1)
- Immersion (1)
- Immersive Environments (1)
- Information systems (1)
- Innovationsmanagement (1)
- Interaction Techniques (1)
- Interaction design process and methods (1)
- Interaction techniques (1)
- Interactivity (1)
- Interaktion (1)
- Interaktionsmodellierung (1)
- Kinect (1)
- LIDAR (1)
- Large screen displays (1)
- Loudspeakers (1)
- MPEG-4 (1)
- Markteinführungskonzept (1)
- Mobile Computing (1)
- Mobile computing (1)
- Mulitmodal (1)
- Multimedia systems (1)
- Myo (1)
- Navigation (1)
- Nondestructive Testing (1)
- OSC mapping (1)
- Photospherical Imagery (1)
- Preisabsatzfunktion (1)
- Production (1)
- Rapid Prototyping (1)
- Remote collaboration (1)
- Reverberation (1)
- Room effects (1)
- Simulation biegeschlaffer Objekte (1)
- Size control (1)
- Skeleton (1)
- Spatial media (1)
- SteamVR (1)
- Stereo camera (1)
- Studio (1)
- Szenenanalyse (1)
- Testmarkt (1)
- Testumgebung (1)
- UNIX (1)
- Ultrasonic (1)
- Verfolgen von Darstellern (1)
- Vibrotaktiles Feedback (1)
- Video Conferencing (1)
- Video Texturen (1)
- Virtual Concerts (1)
- Virtual environments (1)
- Virtual reality (1)
- Visualisierung (1)
- Volumenbestimmung (1)
- Zweihändige Eingabe (1)
- abstraction-reconstruction cycle (1)
- algorithmus (1)
- application streaming (1)
- audio filtering (1)
- audio interpolation (1)
- audio signal processing, audio telecommunications (1)
- augmented (1)
- auralization (1)
- automated robotic camera system (1)
- automated shot control (1)
- autonomous actors (1)
- body tracking (1)
- calm technology (1)
- camera-tracking (1)
- capacitive touch screens (1)
- chatspaces (1)
- collaborative work (1)
- computer education (1)
- computer music (1)
- database (1)
- databases (1)
- design (1)
- dictionary (1)
- digital broadcasting (1)
- direction signalization (1)
- directional mixing console (1)
- distance cue (1)
- eLearning (1)
- erweiterte Realität (1)
- fiducial tangible recognition (1)
- film rules (1)
- graphics processing unit (1)
- human computer interaction (1)
- hypertext, search engine (1)
- interaction in virtual sets (1)
- interactive sport simulation (1)
- level of detail (1)
- light interaction (1)
- live streaming (1)
- loudspeaker array (1)
- low cost multi-touch infrared overlay frame (1)
- man-machine interfaces (1)
- marketing studies (1)
- measurement (1)
- mixed reality (1)
- mobile computing (1)
- multi-point interfaces (1)
- multichannel sound reproduction (1)
- multimedia (1)
- multimedia information systems (1)
- musical expression (1)
- narrowcasting functions (1)
- navigation aids (1)
- networked audio (1)
- obtrusion (1)
- perspicuity (1)
- photonic mixing device (1)
- photospherical imagery (1)
- planning tools (1)
- post production (1)
- presentation assistance (1)
- psychoacoustic (1)
- realtime animations (1)
- realtime rendering (1)
- reliability (1)
- retrieval methods (1)
- room-related transfer functions (1)
- roomware (1)
- rotational tracking (1)
- scene analysis (1)
- shader (1)
- shader programming (1)
- sketching (1)
- social media (1)
- soft shadow (1)
- sound interaction (1)
- sound localization (1)
- sound radiation pattern (1)
- sound spatialization resource management (1)
- sound spatialization server (1)
- spacecraft configuration (1)
- spatial sound (1)
- spatial sound design (1)
- stage flight system (1)
- switcher control (1)
- talent tracking (1)
- tangibles (1)
- telepresence (1)
- user experience (1)
- user interface design man-machine interfaces (1)
- user-centered design (1)
- verification (1)
- vibrotactile feedback (1)
- video textures (1)
- virtual TV (1)
- virtual conferencing environment (1)
- virtual realities (1)
- virtual reality (1)
- virtual reality environments (1)
- virtual studio interaction (1)
- virtual studios (1)
- visual feedback (1)
- visual music (1)
- vizualization (1)
- vr archery (1)
- wayfinding (1)
Department/institution
- Creative Media Production and Entertainment Computing (89) (remove)
Actors in virtual studio productions are faced with the challenge that they have to interact with invisible virtual objects because these elements are rendered separately and combined with the real image later in the production process. Virtual sets typically use static virtual elements or animated objects with predefined behavior so that actors can practice their performance and errors can be corrected in the post production. With the demand for inexpensive live recording and interactive TV productions, virtual objects will be dynamically rendered at arbitrary positions that cannot be predicted by the actor. Perceptive aids have to be employed to support a natural interaction with these objects. In our work we study the effect of haptic feedback for a simple form of interaction. Actors are equipped with a custom built haptic belt and get vibrotactile feedback during a small navigational task (path following). We present a prototype of a wireless vibrotactile feedback device and a small framework for evaluating haptic feedback in a virtual set environment. Results from an initial pilot study indicate that vibrotactile feedback is a suitable non-visual aid for interaction that is at least comparable to audio-visual alternatives used in virtual set productions.
Live video streaming is becoming increasingly popular as a form of interaction in social applications. One of its main advantages is an ability to immediately create and connect a community of remote users on the spot. In this paper we discuss how this feature can be used for crowdsourced completion of simple visual search tasks (such as finding specific objects in libraries and stores, or navigating around live events) and social interactions through mobile mixed reality telepresence interfaces. We present a prototype application that allows users to create a mixed reality space with a photospherical imagery as a background and interact with other connected users through viewpoint, audio, and video sharing, as well as realtime annotations in mixed reality space. Believing in the novelty of our system, we conducted a short series of interviews with industry professionals on the possible applications of our system. We discuss proposed use-cases for user evaluation, as well as outline future extensions of our system.
A visual and spatial feedback system for orientation in virtual sets of virtual TV studios was developed and evaluated. It is based on a green proxy object, which moves around in the acting space by way of four transparent wires. A separate unit controls four winches and is connected to an engine, which renders the virtual set. A new developed plugin registers a virtual object’s position with the proxy object which imitates the virtual object’s movement on stage. This will allow actors to establish important eye contact with a virtual object and feel more comfortable in a virtual set. Furthermore, interaction with the virtual object and its proxy can be realised through a markerless actor tracking system. Several possible scenarios for user application were recorded and presented to experts in the broadcast industry, who evaluated the potential of SpiderFeedback in interviews and by questionnaires.
Virtual environment walkthrough applications are generally enhanced by a user’s interactions within a simulated architectural space, but the enhancement that stems from changes in spatial sound that are coupled with a user’s behavior are particularly important, especially within regard to creating a sense of place. When accompanied by stereoscopic image synthesis, spatial sound can immerse the user in a high-realism virtual copy of the real world. An advanced virtual environment that allow users to change realtime rendering features with a few manipulations has been shown to enable switching between different versions of a modeled space while maintaining sensory immersion. This paper reports on an experimental project in which an architectural model is being integrated into such an interactive virtual environment. The focus is on the spatial sound design for supporting interaction, including demonstrations of both the possibilities and limitations of such applications in presenting and promoting architectural designs, as well as in three-dimensional sketching.
Using spatial audio successfully for augmented reality (AR) applications is a challenge, but is awarded with an improved user experience. Thus, we have extended the AR/VR framework \sc Morgan with spatial audio to improve users orientation in an AR application. In this paper, we investigate the users’ capability to localize and memorize spatial sounds (registered with virtual or real objects). We discuss two scenarios. In the first scenario, the user localizes only sound sources and in the second scenario the user memorizes the location of audio-visual objects. Our results reflect spatial audio performance within the application domain and show which technology pitfalls still exist. Finally, we provide design recommendations for spatial audio AR environments.
In a virtual reality environment users are immersed in a scene with objects which might produce sound. The responsibility of a VR environment is to present these objects, but a system has only limited resources, including spatialization channels (mixels), MIDI/audio channels, and processing power. The sound spatialization resource manager controls sound resources and optimizes fidelity (presence) under given conditions. For that a priority scheme based on human psychophysical hearing is needed. Parameters for spatialization priorities include intensity calculated from volume and distance, orientation in the case of non-uniform radiation patterns, occluding objects, frequency spectrum (low frequencies are harder to localize), expected activity, and others. Objects which are spatially close together (depending on distance and direction) can be mixed. Sources that can not be spatialized can be treated as a single ambient sound source. Important for resource management is the resource assignment, i.e., minimizing swap operations, which makes it desirable to look-ahead and predict upcoming events in a scene. Prediction is achieved by monitoring objects’ speed and past evaluation values. Fidelity is contrasted for Zifferent kind of resource restrictions and optimal resource assignment based upon unlimited dynamic scene look-ahead. To give standard and comparable results, the VRML 2.0 specification is used as an application programmer interface. Applicability is demonstrated with a helical keyboard, a polyphonic MIDI stream driven animation including user interaction (user moves around, playing together with programmed notes). The developed sound spatialization resource manager gives improved spatialization fidelity under runtime constraints. Application programmers and virtual reality scene designers are freed from the burden of assigning and predicting the sound sources.
The Sound Spatialization Framework is a C++ toolkit and development environment for providing advanced sound spatialization for virtual reality and multimedia applications. The Sound Spatialization Framework provides many powerful display and user-interface features not found in other sound spatialization software packages. It provides facilities that go beyond simple sound source spatialization: visualization and editing of the soundscape, multiple sinks, clustering of sound sources, monitoring and controlling resource management, support for various spatialization backends, and classes for MIDI animation and handling.
The Sound Spatialization Framework is a C++ toolkit and development environment for providing advanced sound spatialization for virtual reality and multimedia applications. The Sound Spatialization Framework provides many powerful display and user-interface features not found in other sound spatialization software packages. It provides facilities that go beyond simple sound source spatialization: visualization and editing of the soundscape, multiple sinks, clustering of sound sources, monitoring and controlling resource management, support for various spatialization backends, and classes for MIDI animation and handling.
Keywords:
sound spatialization, resource management, virtual environments, spatial sound authoring, user interface design, human-machine interfaces
With the virtual environment developed here, the characteristic
sound radiation patterns of musical instruments can be experienced
in real-time. The user may freely move around a musical instrument, thereby receiving acoustic and visual feedback in real-time. The perception of auditory and visual effects is intensified by the combination of acoustic and visual elements, as well as the option of user interaction. The simulation of characteristic sound radiation patterns is based on interpolating the intensities of a multichannel recording and offers a near-natural mapping of the sound radiation patterns. Additionally, a simple filter has been developed, enabling the qualitative simulation of an instrument’s characteristic sound radiation patterns to be easily implemented within real-time 3D applications. Both methods of simulating sound radiation patterns have been evaluated for a saxophone with respect to their functionality and validity by means of spectral analysis and an auditory experiment.
Virtual TV studios use actor tracking systems for resolving the occlusion of computer graphics and studio camera image. The actor tracking delivers the distance between actor and studio camera. We deploy a photonic mixing device, which captures a depth map and a luminance image at low resolution. The renderer engines gets one depth value per actor using the OSC protocol. We describe the actor recognition algorithm based on the luminance image and the depth value calculation. We discuss technical issues like noise and calibration.
Live video broadcasting requires a multitude of professional expertise to enable multi-camera productions. Robotic systems allow the automation of common and repeated tracking shots. However, predefined camera shots do not allow quick adjustments when required due to unpredictable events. We introduce a modular automated robotic camera control and video switch system, based on fundamental cinematographic rules. The actors' positions are provided by a markerless tracking system. In addition, sound levels of actors' lavalier microphones are used to analyse the current scene. An expert system determines appropriate camera angles and decides when to switch from one camera to another. A test production was conducted to observe the developed prototype in a live broadcast scenario and served as a video-demonstration for an evaluation.
In this paper we describe a prototypical system for live musical performance in a virtual studio environment. The performer stands in front of the studio camera and interacts with an infrared-laser-based multi-touch device. The final TV image shows the performer interacting with a virtual screen which is augmented in front of herself. To overcome the problem of the performer not seeing this virtual screen in reality, we use a special hexagonal grid to facilitate the performer's awareness of this novel Theremin-like virtual musical instrument.
Given limited computational resources available for the rendering of spatial sound imagery, we seek to determine effective means for choosing whatcomponents of the rendering will provide the most audible differences in the results. Rather than begin with an analytic approach that attempts to predict audible differences on the basis of objective parameters, we chose to begin with subjective tests of how audibly different the rendering result may be heard to be when that result includes two types of sound obstruction: reflectors and occluders. Single-channel recordings of 90 short speech sounds were made in an anechoic chamber in the presence and absence of these two types of obstructions, and as the angle of those obstructions varied over a 90 degree range. These recordings were reproduced over a single loudspeaker in that anechoic chamber, and listeners were asked to rate how confident they were that the recording of each of these 90 stimuli included an obstruction. These confidence ratings can be used as an integral component in the evaluation function used to determine which reflectors and occluders are most important for rendering.
Level-of-detail is a concept well-known in computer graphics to reduce the number of rendered polygons. Depending on the distance to the subject (viewer), the objects’ representation is changed. A similar concept is the clustering of sound sources for sound spatialization. Clusters can be used to hierarchically organize mixels and to optimize the use of resources, by grouping multiple sources together into a single representative ource. Such a clustering process should minimize the error of position allocation of
elements, perceived as angle and distance, and also differences between velocity relative to the sink (i.e., Doppler shift). Objects with similar direction of motion and speed (relative to sink) in the same acoustic resolution cone and with similar distance to a sink can be grouped together.
Level-of-detail is a concept well-known in computer graphics to reduce the number of rendered polygons. Depending on the distance to the subject (viewer), the objects’ representation is changed. A similar concept is the clustering of sound sources for sound spatialization. Clusters can be used to hierarchically organize mixelsand to optimize the use of resources, by grouping multiple sources together into a single representative source. Such a clustering process should minimize the error of position allocation of elements, perceived as angle and distance, and also differences between velocity relative to the sink (i.e., Doppler shift). Objects with similar direction of motion and speed (relative to sink) in the same acoustic resolution cone and with similar distance to a sink can be grouped together.
The interpretation process of complex data sets makes the integration of effective interaction techniques crucial. Recent work in the field of human-computer interaction has shown that there is strong evidence that multimodal user interaction, i.e. the integration of various input modalities and interaction techniques into one comprehensive user interface, can improve human performance when interacting with complex data sets. However, it is still unclear which factors make these user interfaces superior to unimodal user interfaces. The contribution of this work is an analytical comparison of a multimodal and a unimodal user interface for a scientific visualization application. We show that multimodal user interaction with simultaneously integrated speech and gesture input improves user performance regarding efficiency and ease of use.
The article discusses the question of “How to convey the experience in a virtual environment to third parties?” and explains the different technical implementations which can be used for live streaming and recording of a mixed reality experience. The real-world applications of our approach include education, entertainment, e- sports, tutorials, and cinematic trailers, which can benefit from our research by finding a suitable solution for their needs. We explain and outline our Mixed Reality systems as well as discuss the experience of recorded demonstrations of different VR applications, including the need for calibrated camera lens parameters based on realtime encoder values.
We report about several experiments on applying mixed reality technology in the context of accessing collective memories from atomic bombs, Holocaust and Second World War. We discuss the impact of Virtual Reality, Augmented Virtuality and Augmented Reality for specific memorial locations. We show how to use a virtual studio for demonstrating an augmented reality application for a specific location in a remote session within a video conference. Augmented Virtuality is used to recreate the local environment, thus providing a context and helping the participants recollect emotions related to a certain place. This technique demonstrates the advantages of using virtual (VR) and augmented (AR) reality environments for rapid prototyping and pitching project ideas in a live remote setting.