Refine
Year of publication
Document Type
- Conference Proceeding (118)
- Article (19)
- Part of a Book (15)
- Diploma Thesis (3)
- Article trade magazine (2)
- Doctoral Thesis (1)
- Periodical (1)
Language
- English (142)
- German (14)
- Multiple languages (3)
Keywords
- VSVR (46)
- Virtual (TV) Studio (23)
- FHD (22)
- Sound Spatialization (8)
- Resource Management (6)
- Mixed Reality (5)
- Poster (5)
- audio rendering (5)
- human perception (5)
- virtual environments (5)
Department/institution
- Creative Media Production and Entertainment Computing (159) (remove)
Einfach benutzbare VR-Anwendungen erfordern andere Interaktionstechniken als konventionelle Desktop-Anwendungen mit Maus, Tastatur und Desktop-Metapher zur Verfügung stellen. Da solche Ansätze in Konzeption und Realisierung deutlicher komplexer sind, müssen diese mit Sorgfalt ausgewählt werden. Folgt man der Argumentation, dass VR eine natürliche Interaktion mit virtuellen Objekten ermöglicht, so führt dies fast zwangsläufig zu zweihändigen Interaktionstechniken für virtuelle Umgebungen, da Benutzer in realen Umgebungen gewohnt sind, fast ausschlieï‚lich zweihändig zu agieren. In diesem Beitrag geben wir eine Übersicht über den Stand der Technik im Bereich zweihändiger Interaktion, leiten Anforderungen an eine Entwicklung zweihändiger Interaktionstechniken in VR ab und beschreiben einen eigenen Ansatz. Dabei geht es um die zweihändige Interaktion bei der Simulation flexibler biegeschlaffer Bauteile (z. B. Schlauchverbindungen).
A module for soundscape monitoring and visualizing resource management processes was extended for presenting clusters, generated by a novel sound source clustering algorithm. This algorithm groups multiple sound sources together into a single representative source, considering localization errors depending on listener orientation. Localization errors are visualized for each cluster using resolution cones. Visualization is done in runtime and allows understanding and evaluation of the clustering algorithm.
A module for soundscape monitoring and visualizing resource management processes was extended for presenting clusters, generated by a novel sound source clustering algorithm. This algorithm groups multiple sound sources together into a single representative source, considering localization errors depending on listener orientation. Localization errors are visualized for each cluster using resolution cones. Visualization is done in runtime and allows understanding and evaluation of the clustering algorithm.
Dynamic characteristics of occlusion during lower jaw motion are useful in the diagnosis of jaw articulation problems and in computer-aided design/manufacture of teeth restorations. The Functionally Generated Path (FGP), produced as a surface which envelops the actual occlusal surface of the moving opponent jaw, can be used for compact representation of dynamic occlusal relations. In traditional dentistry FGP is recorded as a bite impression in a patient’s mouth. We propose an efficient computerized technique for FGP reconstruction and validate it through implementation and testing. The distance maps between occlusal surfaces of jaws, calculated for multiple projection directions and accumulated for mandibular motion, provide information for FGP computation. Rasterizing graphics hardware is used for fast calculation of the distance maps. Real-world data are used: the scanned shape of teeth and the measured motion of the lower jaw. We show applications of FGP to analysis of the occlusion relations and occlusal surface design for restorations.
CAD/CAM techniques are increasingly used in dentistry for the design and fabrication of teeth estorations. Important concerns are the correction of articulation problems that existed before treatment and the prevention of treatment-generated problems. These require interactive evaluation of the occlusal surfaces of teeth during mastication. Traditional techniques based on the use of casts with mechanical articulators require manual adjustment of occlusal surfaces, which becomes impractical when hard restoration materials like porcelain are used; they are also time and labor consuming and provide little visual information. We present new visual tools and a related user interface for global articulation simulation, developed for the Intelligent Dental Care System project. The aim of the simulation is visual representation of characteristics relevant to the chewing process. The simulation is based on the construction of distance maps, which are visual representations of the distributions of the distances of points in a tooth to the opposite jaw. We use rasterizing graphics hardware for fast calculation of the distance maps. Distance maps are used for collision detection and for the derivation of various characteristics showing the distribution of load on the teeth and the chewing capability of the teeth. Such characteristics can be calculated for particular positions of the jaws; cumulative characteristics are used to describe the properties of jaw movement. This information may be used for interactive design of the occlusal surfaces of restorations and for jaw articulation diagnosis. We also demonstrate elements of a user interface that exploit metaphors familiar to dentists from everyday practice.
CAD/CAM techniques are increasingly used in dentistry for the design and fabrication of teeth restorations. Important concerns are the correction of articulation problems that existed beforetreatment and the prevention of treatment-generated problems. These require interactive evaluation of the occlusal surfaces of teeth during mastication. Traditional techniques based on the use of casts with mechanical articulators require manual adjustment of occlusal surfaces, which becomes impractical when hard restoration materials like porcelain are used; they are also time and labor consuming and provide little visual information. We present new visual tools and a related user interface for global articulation simulation, developed for the Intelligent Dental Care System project. The aim of the simulation is visual representation of characteristics relevant to the chewing process. The simulation is based on the construction of distance maps, which are visual representations of the distributions of the distances of points in a tooth to the opposite jaw. We use rasterizing graphics hardware for fast calculation of the distance maps. Distance maps are used for collision detection and for the derivation of various characteristics showing the distribution of load on the teeth and the chewing capability of the teeth. Such characteristics can be calculated for particular positions of the jaws; cumulative characteristics are used to describe the properties of jaw movement. This information may be used for interactive design of the occlusal surfaces of restorations and for jaw articulation diagnosis. We also demonstrate elements of a user interface that exploit metaphors familiar to dentists from everyday practice.
Shadows in computer graphics are an important rendering aspect for spatial objects. For realtime computer applications such as games, it is essential to represent shadows as accurate as possible. Also, various tv stations work with virtual studio systems instead of real studio sets. Especially for those systems, a realistic impression of the rendered and mixed scene is important. One challenge, hence, is the creation of a natural shadow impression. This paper presents the results of an empirical study to compare the performance and quality of different shadow mapping methods. For this test, a prototype studio renderer was developed. A percentage closer filter (pcf) with a number of specific resolutions is used to minimize the aliasing issue. More advanced algorithms which generate smooth shadows like the percentage closer soft shadow (pcss) method as well as the variance shadow maps (vsm) method are analysed. Different open source apis are used to develop the virtual studio renderer, giving the benefit of permanent enhancement. The Ogre 3D graphic engine is used to implement the rendering system, benefiting from various functions and plugins. The transmission of the tracking data is accomplished with the vrpn server/client and the Intersense api. The different shadow algorithms are compared in a virtual studio environment which also casts real shadows and thus gives a chance for a direct comparison throughout the empirical user study. The performance is measured in frames per secon
Der neunte Workshop "Virtuelle und Erweiterte Realität" der Fachgruppe VR/AR der Gesellschaft für Informatik e.V. >> wurde an der FH Düsseldorf vom 19.09. - 20.09.2012 durchgeführt. Dies ist der Tagungsband des neunten Workshops zum Thema "Virtuelle und Erweiterte Realität", die von der Fachgruppe VR/AR der Gesellschaft für Informatik e.V. ins Leben gerufen wurde. Als etablierte Plattform für den Informations- und Ideenaustausch der deutschsprachigen VR/AR-Szene bot der Workshop den idealen Rahmen, aktuelle Ergebnisse und Vorhaben aus Forschung und Entwicklung – auch provokative Ideen – im Kreise eines fachkundigen Publikums zur Diskussion zu stellen. Insbesondere wollten wir auch jungen Nachwuchswissenschaftlern die Möglichkeit geben, ihre Arbeiten zu präsentieren; darunter fielen laufende Promotionsvorhaben oder auch herausragende studentische Arbeiten von Hochschulen und Forschungseinrichtungen. Eine Auswahl der besten Artikel aus dem Workshop werden in einer Sonderausgabe des Journal of Virtual Reality and Broadcasting (JVRB) veröffentlicht.
Der fünfzehnte Workshop "Virtuelle und Erweiterte Realität" der Fachgruppe VR/AR der Gesellschaft für Informatik e.V. wurde an der Hochschule Düsseldorf vom 10.-11.10.2018 durchgeführt. Als etablierte Plattform für den Informations- und Ideenaustausch der deutschsprachigen VR/AR-Szene bot der Workshop den idealen Rahmen, aktuelle Ergebnisse und Vorhaben aus Forschung und Entwicklung – auch provokative Ideen – im Kreise eines fachkundigen Publikums zur Diskussion zu stellen. Insbesondere wurde auch jungen Nachwuchswissenschaftlern die Möglichkeit gegeben, ihre Arbeiten zu präsentieren; darunter fielen laufende Promotionsvorhaben oder auch herausragende studentische Arbeiten von Hochschulen und Forschungseinrichtungen. Das Programmkomitee hat 12 Lang- und 6 Kurzbeiträge aus über 29 Gesamteinreichungen ausgewählt. Die Beiträge decken das Spektrum der Virtuellen und Erweiterten Realität ab. Der Workshop fand am 10. Oktober gleichzeitig mit dem Innovationstag des Innovationshubs in Düsseldorf statt. Für einen Beitrag eigneten sich alle Themenfelder der "Virtuellen und Erweiterten Realität", insbesondere:
3D Eingabegeräte und Interaktionstechniken
Avatare und Agenten
Displaytechnologien und Tracking
(Echtzeit-)Rendering
Education und Edutainment
Entertainment und Experiences
Gesellschaft und soziotechnische Aspekte
Human Factors
Industrielle Einsatzszenarien
Innovative Anwendungen
Künstlerische Anwendungen
Modellierung und Simulation
Multimodale Interaktion
Systemarchitekturen und Intelligente Umgebungen
Verteilte und kooperative VR/AR-Umgebungen
For this study, an experimental vibrotactile feedback system was developed to help actors with the task of moving their arm to a certain place in a virtual tv studio under live conditions. Our intention is to improve interaction with virtual objects in a virtual set, which are usually not directly visible to the actor, but only on distant displays. Vibrotactile feedback might improve the appearance on tv because an actor is able to look in any desired direction (camera or virtual object) or to read text on a teleprompter while interacting with a virtual object. Visual feedback in a virtual studio lacks spatial relation to the actor, which impedes the adjustment of the desired interaction. The five tactors of the implemented system which are mounted on the tracked arm give additional information like collision, navigation and activation. The user study for the developed system shows that the duration for reaching a certain target is much longer in case no visual feedback is given, but the accuracy is similar. In this study, subjects reported that an activation signal indicating the arrival at the target of a drag & drop task was helpful. In this paper, we discuss the problems we encountered while developing such a vibrotactile display. Keeping these pitfalls in mind could lead to better feedback systems for actors in virtual studio environments.
Die Visualisierung von Produkten in Echtzeit ist in vielen Bereichen ein hilfreicher Schritt, um potentiellen Kunden eine Vorstellung vom Einsatzgebiet und einen Überblick über die finale Anwendung zu erlauben. In den letzten Jahren haben neue Technologien in der Grafikkartenindustrie dazu geführt, dass früher nur auf teuren Grafikworkstations verfügbare Möglichkeiten nun auch mit relativ kostengünstigen Karten, welche für den Einsatz in Standard-PCs konzipiert wurden, realisierbar sind.
Es wird an einem Modellentwurf des Innenraums des People Cargo Movers gezeigt, wie die Beleuchtung innerhalb einer Echtzeitvisualisierung durch Shader realisiert werden kann. Als Lichtquelle wird dabei eine Landschaftsaufnahme herangezogen, welche als eine von mehreren Videotexturen eingebunden wurde. Außerdem werden real im virtuellen Studio gefilmte Personen im Innenraum gleichermaï‚en über Videotexturen dargestellt und ebenfalls durch die Landschaft beleuchtet.
Für moderne interaktive Anwendungen wird es immer wichtiger, einen Benutzer durch zusätzlichen Ballast so wenig wie möglich einzuschränken. Daher bietet sich eine kamerabasierte Interaktionserkennung an. Viele existierende Verfahren benötigen dazu aber einen weitestgehend statischen Hintergrund. In der gegebenen Anwendung allerdings befindet sich im Sichtbereich der Kameras eine Projektion bewegter Inhalte, mit denen ein Benutzer interagieren kann. Im Rahmen dieser Arbeit sollte ein bestehendes, auf Infrarotlicht basierendes System verbessert werden, das sich bisher als beleuchtungsabhängig erwiesen hat. Dazu wurden zunächst mehrere Verfahren zur Trennung von Vorder- und Hintergrund auf ihre Tauglichkeit untersucht. Das favorisierte Verfahren sollte anschließend durch stereoskopische Bildaufnahme - und damit tiefenbasierte Trennung - so verbessert werden, dass der bewegte Hintergrund sicher unterdrückt wird. Dies erwies sich für die gewünschten Anwendungen zwar als nicht praktikabel, dennoch wurden mögliche andere Anwendungsbereiche gefunden. Auch die Untersuchung der einzelnen Trennungsverfahren lieferte ein Ergebnis, das - mit weiteren Tests - zur Verbesserung der bestehenden Installation beitragen kann.
Virtual environments can create a realistic impression of an architectural space during the architectural design process, providing a powerful tool for evaluation and promotion during a project’s early stages. In comparison to pre-rendered animations,
such as walkthroughs based on CAD models, virtual environments can offer intuitive interaction and a more life like experience. Advanced virtual environments allow users to change realtime rendering features with a few manipulations, switching between different versions while still maintaining sensory immersion. This paper reports on an experimental project in which architectural models are being integrated into interactive virtual environments, and includes demonstrations of both the possibilities and limitations of such applications in evaluating, presenting and promoting architectural designs.
More than three decades of ongoing research in immersive modelling has revealed many advantages of creating objects in virtual environments. Even though there are many benefits, the potential of immersive modelling has only been partly exploited due to unresolved problems such as ergonomic problems, numerous challenges with user interaction and the inability to perform exact, fast and progressive refinements. This paper explores past research, shows alternative approaches and proposes novel interaction tools for pending problems. An immersive modelling application for polygon meshes is created from scratch and tested by professional users of desktop modelling tools, such as Autodesk Maya, in order to assess the efficiency, comfort and speed of the proposed application with direct comparison to professional desktop modelling tools.
This paper presents a mobile approach of integrating tangible user feedback in today’s virtual TV studio productions. We describe a tangible multitouch planning system, enabling a single user to prepare and customize scene flow and settings. Users can view and interact with virtual objects by using a tangible user interface on a capacitive multitouch surface. In a 2D setting created TV scenes are simultaneously rendered as separate view using a production/target renderer in 3D. Thereby the user experiences a closer reproduction of a final production and set assets can be reused. Subsequently, a user can arrange scenes on a timeline while maintaining different versions/sequences. The system consists of a tablet and a workstation, which does all application processing and rendering. The tablet is just an interface connected via wireless LAN.
This article presents a new approach of integrating tangible user feedback in todays virtual TV studio productions. We describe a tangible multitouch planning system, enabling multiple users to prepare and customize scene flow and settings. Users can collaboratively view and interact with virtual objects by using a tangible user interface on a shared multitouch surface. The in a 2D setting created TV scenes are simultaneously rendered on an external monitor, using a production/target renderer in 3D. Thereby the user experiences a closer reproduction of a final production. Subsequently, users are able to join together the scenes into one complex plot. Within the developing process, a video prototype of the system shows the user interaction and enables early reviews and evaluations. The requirement analysis is based on expert interviews.
Broader use of virtual reality environments and sophisticated animations spawn a need for spatial sound. Until now, spatial sound design has been based very much on experience and trial and error. Most effects are hand-crafted, because good design tools for spatial sound do not exist. This paper discusses spatial sound authoring and its applications, including shared virtual reality environments based on VRML. New utilities introduced by this research are an inspector for sound sources, an interactive resource manager, and a visual soundscape manipulator. The tools are part of a sound spatialization framework and allow a designer/author of multimedia content to monitor and debug sound events. Resource constraints like limited sound spatialization channels can also be simulated.
The Soundtrack Of Your Life
(2012)
The PSFC, or Pioneer sound field control system, is a DSP-driven hemispherical 14-loudspeaker array, installed at the University of Aizu Multimedia Center. Collocated with a large screen rear-projection stereographic display the PSFC features realtime control of virtual room characteristics and direction of two separate sound channels, smoothly steering them around a configurable soundscape. The PSFC controls an entire sound field, including sound direction, virtual distance, and simulated environment (reverb level, room size and liveness) for each source. It can also configure a dry (DSP-less) switching matrix for direct directionalization. The PSFC speaker dome is about 14 m in diameter, allowing about twenty users at once to comfortably stand or sit near its sweet spot.
Auditory displays with the ability to dynamically spatialize virtual sound sources under real-time conditions enable advanced applications for art and music. A listener can be deeply immersed while interacting and participating in the experience. We review some of those applications while focusing on the Helical Keyboard project and discussing the required technology. Inspired by the cyclical nature of octaves and helical structure of a scale, a model of a piano-style keyboard was prepared, which was then geometrically warped into a helicoidal configuration, one octave/revolution, pitch mapped to height and chroma. It can be driven by MIDI events, real-time or sequenced, which stream is both synthesized and spatialized by a spatial sound display. The sound of the respective notes is spatialized with respect to sinks, avatars of the human user, by default in the tube of the helix. Alternative coloring schemes can be applied, including a color map compatible with chromastereoptic eyewear. The graphical display animates polygons, interpolating between the notes of a chord across the tube of the helix. Recognition of simple chords allows directionalization of all the notes of a major triad from the position of its musical root. The system is designed to allow, for instance, separate audition of harmony and melody, commonly played by the left and right hands, respectively, on a normal keyboard. Perhaps the most exotic feature of the interface is the ability to fork oneÃs presence, replicating subject instead of object by installing multiple sinks at arbitrary places around a virtual scene so that, for example, harmony and melody can be separately spatialized, using two heads to normalize the octave; such a technique effectively doubles the helix from the perspective of a single listener. Rather than a symmetric arrangement of the individual helices, they are perceptually superimposed in-phase, co-extensively, so that corresponding notes in different registers are at the same azimuth.
The conductor's philosophy
(2013)
Multimediale Technologien werden in der Marktforschung immer stärker eingesetzt, um flexible und kostengünstige Studien durchzuführen. Im Innovationsprozess kann dabei auf die langjährigen Erfahrungen zurückgegriffen werden, die durch den Einsatz der Computersimulation in der technischen Produktentwicklung zustande gekommen sind. In sehr frühen Phasen des Innovationsprozesses können durch Einsatz der neuen Technologien die Markteinführungskonzepte für neue Produkte getestet werden. Die Applikationen der virtuellen Realität bieten ein einzigartiges Potential, neue Produkte einschlieï‚lich des Marketingkonzeptes zu testen, ohne dass dieses Produkt bereits physisch vorhanden sein muss. Am Beispiel eines Elementes des Marketingkonzeptes, der Preispolitik, zeigt die vorliegende Studie auf, welches Potential die virtuelle Kaufsituation von Produkten bietet. Der Fokus des Projektes liegt auf der interaktiven Produktpräsentation in einer virtuellen Umgebung, die in eine Online-Befragung mit zusätzlichen Werbefilmen eingebettet ist. Visuell hochwertige 3D-Produktpräsentationen versetzen den Probanden in eine virtuelle Einkaufsumgebung, die einem realen Szenario entspricht. Die virtuellen Produkte werden in mehreren Kaufentscheidungsrunden zu unterschiedlichen Preisen angeboten. Der Preisuntersuchung geht eine Präsentation ausgewählter Werbespots sowie eine produktbezogene Befragung voraus. Im Anschluss an die virtuellen Preisentscheidungen werden die Eindrücke sowie einige Kontrollgröen abgefragt. In weitergehenden Studien dieser Art können die Wirkungen mehrerer Marketing-Instrumente zu einem Zeitpunkt untersucht werden, in dem sich die Produkte noch im Entwicklungsprozess befinden. Auf diesem Weg lassen sich auch Wettbewerbsvorteile bestehender Produkte effizienter erkennen und nutzen. Mit den hoch entwickelten Computer- und Visualisierungstechnologien ist ein mächtiges Werkzeug entstanden, das bereits für kommerzielle Präsentationen und Produktstudien eingesetzt wird. Zukünftig kann es auch in Kombination mit Internetanwendungen und klassischen Methoden der Marktforschung zu einem sehr frühen Zeitpunkt umfassende Erkenntnisse über ein Produkt liefern.
Markerless talent tracking is widely used for interactions and animations within virtual environments. In a virtual (tv) studio talents could be overburden by interaction tasks because camera and text require extensive attention. We take a look into animations and inter- actions within a studio, which do not require any special attention or learning. We show the generation of an artificial shadow from a talent, which ease the keying process, where separation of real shadows from the background is a difficult task. We also demonstrate animations of footsteps and dust. Furthermore, capturing talents’ height can also be used to adjust the parameters of elements in the virtual environment, like the position and scaling of a virtual display. In addition to the talents, a rigid body was tracked as placeholder for graphics, easing the interaction tasks for a talent. Two test productions show the possibilities, which subtle animations offer. In the second production, the rendering was improved (shadows, filtering, normal maps, ...) and instead of using the rigid body to move an object (a flag), the animation was only controlled by the hand’s position.
Actors in virtual studio productions are faced with the challenge that they have to interact with invisible virtual objects because these elements are rendered separately and combined with the real image later in the production process. Virtual sets typically use static virtual elements or animated objects with predefined behavior so that actors can practice their performance and errors can be corrected in the post production. With the demand for inexpensive live recording and interactive TV productions, virtual objects will be dynamically rendered at arbitrary positions that cannot be predicted by the actor. Perceptive aids have to be employed to support a natural interaction with these objects. In our work we study the effect of haptic feedback for a simple form of interaction. Actors are equipped with a custom built haptic belt and get vibrotactile feedback during a small navigational task (path following). We present a prototype of a wireless vibrotactile feedback device and a small framework for evaluating haptic feedback in a virtual set environment. Results from an initial pilot study indicate that vibrotactile feedback is a suitable non-visual aid for interaction that is at least comparable to audio-visual alternatives used in virtual set productions.
Live video streaming is becoming increasingly popular as a form of interaction in social applications. One of its main advantages is an ability to immediately create and connect a community of remote users on the spot. In this paper we discuss how this feature can be used for crowdsourced completion of simple visual search tasks (such as finding specific objects in libraries and stores, or navigating around live events) and social interactions through mobile mixed reality telepresence interfaces. We present a prototype application that allows users to create a mixed reality space with a photospherical imagery as a background and interact with other connected users through viewpoint, audio, and video sharing, as well as realtime annotations in mixed reality space. Believing in the novelty of our system, we conducted a short series of interviews with industry professionals on the possible applications of our system. We discuss proposed use-cases for user evaluation, as well as outline future extensions of our system.
A visual and spatial feedback system for orientation in virtual sets of virtual TV studios was developed and evaluated. It is based on a green proxy object, which moves around in the acting space by way of four transparent wires. A separate unit controls four winches and is connected to an engine, which renders the virtual set. A new developed plugin registers a virtual object’s position with the proxy object which imitates the virtual object’s movement on stage. This will allow actors to establish important eye contact with a virtual object and feel more comfortable in a virtual set. Furthermore, interaction with the virtual object and its proxy can be realised through a markerless actor tracking system. Several possible scenarios for user application were recorded and presented to experts in the broadcast industry, who evaluated the potential of SpiderFeedback in interviews and by questionnaires.
Virtual environment walkthrough applications are generally enhanced by a user’s interactions within a simulated architectural space, but the enhancement that stems from changes in spatial sound that are coupled with a user’s behavior are particularly important, especially within regard to creating a sense of place. When accompanied by stereoscopic image synthesis, spatial sound can immerse the user in a high-realism virtual copy of the real world. An advanced virtual environment that allow users to change realtime rendering features with a few manipulations has been shown to enable switching between different versions of a modeled space while maintaining sensory immersion. This paper reports on an experimental project in which an architectural model is being integrated into such an interactive virtual environment. The focus is on the spatial sound design for supporting interaction, including demonstrations of both the possibilities and limitations of such applications in presenting and promoting architectural designs, as well as in three-dimensional sketching.
Using spatial audio successfully for augmented reality (AR) applications is a challenge, but is awarded with an improved user experience. Thus, we have extended the AR/VR framework \sc Morgan with spatial audio to improve users orientation in an AR application. In this paper, we investigate the users’ capability to localize and memorize spatial sounds (registered with virtual or real objects). We discuss two scenarios. In the first scenario, the user localizes only sound sources and in the second scenario the user memorizes the location of audio-visual objects. Our results reflect spatial audio performance within the application domain and show which technology pitfalls still exist. Finally, we provide design recommendations for spatial audio AR environments.
In a virtual reality environment users are immersed in a scene with objects which might produce sound. The responsibility of a VR environment is to present these objects, but a system has only limited resources, including spatialization channels (mixels), MIDI/audio channels, and processing power. The sound spatialization resource manager controls sound resources and optimizes fidelity (presence) under given conditions. For that a priority scheme based on human psychophysical hearing is needed. Parameters for spatialization priorities include intensity calculated from volume and distance, orientation in the case of non-uniform radiation patterns, occluding objects, frequency spectrum (low frequencies are harder to localize), expected activity, and others. Objects which are spatially close together (depending on distance and direction) can be mixed. Sources that can not be spatialized can be treated as a single ambient sound source. Important for resource management is the resource assignment, i.e., minimizing swap operations, which makes it desirable to look-ahead and predict upcoming events in a scene. Prediction is achieved by monitoring objects’ speed and past evaluation values. Fidelity is contrasted for Zifferent kind of resource restrictions and optimal resource assignment based upon unlimited dynamic scene look-ahead. To give standard and comparable results, the VRML 2.0 specification is used as an application programmer interface. Applicability is demonstrated with a helical keyboard, a polyphonic MIDI stream driven animation including user interaction (user moves around, playing together with programmed notes). The developed sound spatialization resource manager gives improved spatialization fidelity under runtime constraints. Application programmers and virtual reality scene designers are freed from the burden of assigning and predicting the sound sources.
The Sound Spatialization Framework is a C++ toolkit and development environment for providing advanced sound spatialization for virtual reality and multimedia applications. The Sound Spatialization Framework provides many powerful display and user-interface features not found in other sound spatialization software packages. It provides facilities that go beyond simple sound source spatialization: visualization and editing of the soundscape, multiple sinks, clustering of sound sources, monitoring and controlling resource management, support for various spatialization backends, and classes for MIDI animation and handling.
The Sound Spatialization Framework is a C++ toolkit and development environment for providing advanced sound spatialization for virtual reality and multimedia applications. The Sound Spatialization Framework provides many powerful display and user-interface features not found in other sound spatialization software packages. It provides facilities that go beyond simple sound source spatialization: visualization and editing of the soundscape, multiple sinks, clustering of sound sources, monitoring and controlling resource management, support for various spatialization backends, and classes for MIDI animation and handling.
Keywords:
sound spatialization, resource management, virtual environments, spatial sound authoring, user interface design, human-machine interfaces
With the virtual environment developed here, the characteristic
sound radiation patterns of musical instruments can be experienced
in real-time. The user may freely move around a musical instrument, thereby receiving acoustic and visual feedback in real-time. The perception of auditory and visual effects is intensified by the combination of acoustic and visual elements, as well as the option of user interaction. The simulation of characteristic sound radiation patterns is based on interpolating the intensities of a multichannel recording and offers a near-natural mapping of the sound radiation patterns. Additionally, a simple filter has been developed, enabling the qualitative simulation of an instrument’s characteristic sound radiation patterns to be easily implemented within real-time 3D applications. Both methods of simulating sound radiation patterns have been evaluated for a saxophone with respect to their functionality and validity by means of spectral analysis and an auditory experiment.
Virtual TV studios use actor tracking systems for resolving the occlusion of computer graphics and studio camera image. The actor tracking delivers the distance between actor and studio camera. We deploy a photonic mixing device, which captures a depth map and a luminance image at low resolution. The renderer engines gets one depth value per actor using the OSC protocol. We describe the actor recognition algorithm based on the luminance image and the depth value calculation. We discuss technical issues like noise and calibration.
Live video broadcasting requires a multitude of professional expertise to enable multi-camera productions. Robotic systems allow the automation of common and repeated tracking shots. However, predefined camera shots do not allow quick adjustments when required due to unpredictable events. We introduce a modular automated robotic camera control and video switch system, based on fundamental cinematographic rules. The actors' positions are provided by a markerless tracking system. In addition, sound levels of actors' lavalier microphones are used to analyse the current scene. An expert system determines appropriate camera angles and decides when to switch from one camera to another. A test production was conducted to observe the developed prototype in a live broadcast scenario and served as a video-demonstration for an evaluation.