Sound and Vibration Engineering
Refine
Year of publication
Document Type
- Conference Proceeding (25)
- Article (20)
- Part of a Book (5)
- Article trade magazine (2)
- Workingpaper / Report (2)
- Bachelor Thesis (1)
- Diploma Thesis (1)
- Researchdata (1)
Language
- English (42)
- German (14)
- Multiple languages (1)
Keywords
- VSVR (14)
- Resource Management (5)
- Sound Spatialization (5)
- FHD (4)
- audio rendering (4)
- human perception (4)
- user interface design (3)
- virtual environments (3)
- DOAJ (2)
- Reverberation (2)
Department/institution
Der Aufführungsraum leistet einen substantiellen Beitrag zum Erleben von Musikdarbietungen. Zur Beschreibung der Akustik von Aufführungsräumen haben sich dabei zahlreiche Messgrößen etabliert, welche bestimmte Wahrnehmungsqualitäten, wie z. B. die Halligkeit, abbilden sollen. Dem stellt jüngere Forschung rein perzeptive Messinventare gegenüber, die den Wahrnehmungseindruck als solchen, losgelöst vom physikalischen Ursprung, beschreiben. Dieser Artikel beschreibt einen Versuch, diese beiden Forschungsrichtungen zusammenzuführen, indem Bewertungen von Aufführungsräumen nach dem perzeptiven Room Acoustical Quality Index (RAQI) durch physikalische Messgrößen modelliert wurden, welche aus einfachen Impulsantwortmessungen ermittelt werden. Dazu wurde eine Auswahl raumakustischer Messgrößen auf fünf Dimensionen reduziert und diese als Prädiktoren für die einzelnen Wahrnehmungsgrößen des RAQI getestet. Die Berechnung dieser Prädiktionsmodelle wird vorgestellt und deren Präzision diskutiert. Anschließend wird die OPRA-Webanwendung vorgestellt, die sowohl die Berechnung der raumakustischen Messgrößen als auch die Prädiktion von RAQI-Wahrnehmungsgrößen für die drei Quellsignale Sprache, Orchester und Solotrompete im Browser ermöglicht. Auch wenn die Präzision der Prädiktionsmodelle noch nicht für alle RAQI-Faktoren zuverlässige Ergebnisse liefert, erlaubt OPRA eine umfangreiche akustische Analyse und eine erste Einschätzung der perzeptiven Wirkung von Aufführungsräumen.
Einsatz und Anpassung von Methoden der Audio-Feature-Extraktion am Beispiel von Indoor Soundscapes
(2022)
Im häuslichen Umfeld erleben Menschen abhängig von ihrer Wohnsituation unterschiedlichste Geräuschumgebungen (Indoor Soundscapes), die einen hohen Einfluss auf das Wohlbefinden haben. Im Rahmen einer Feldstudie wurde daher die theoretisch und praktisch relevante Frage untersucht, welche Merkmale der wahrgenommenen Geräuschumgebungen als besonders ereignisreich oder angenehm bewertet werden. Einhundertfünf Teilnehmer*innen berichteten nach der Experience-Sampling-Methode zeitgesteuert über auftretende Geräuschumgebungen des häuslichen Alltags. Sie bewerteten deren subjektive Wirkung gemäß Soundscape-Standard. Zudem fertigten sie in-situ Audioaufnahmen der Geräuschszenarien an. Die 6594 Tonaufnahmen wurden einer Audioinhaltsanalyse unterzogen und im Zuge dessen wurden vier Featuresets verschiedener Berechnungsansätze extrahiert. Mithilfe der Perzentilen LASSO-Regularisierung wurden lineare gemischte Modelle für Angenehmheit und Ereignisreichtum sowie drei Multilevel-Modelle für Angenehmheit aufgestellt. Die besten Modelle erklären 9 % der Varianz von Angenehmheit und 27 % der Varianz von Ereignisreichtum durch feste Effekte selektierter Prädiktoren aller getesteten Featuresets. Angenehmheit sinkt vor allem, wenn lautheitsbasierte Features hohe Werte zeigen. Ereignisreichtum ist am stärksten abhängig von kurzen Spitzen des C-bewerteten Schalldruckpegels und wird im Vergleich zu Angenehmheit stärker von Zeitschwankungen der Features bestimmt. Durch den Vergleich der Featuresets wird deutlich, dass die Modelleffekte bekannter psychoakustischer Größen durch Hinzufügen von MFCC-Features verbessert werden. Darüber hinaus wird anhand von Multilevel-Modellen gezeigt, dass die Angenehmheit des lautesten Viertels der Tonaufnahmen deutlich besser durch feste Effekte erklärbar ist als der leisere Rest des Datensatzes. Kongruent mit aktuellen Befragungsstudien kann durch Audiofeatures eine Kategorienabhängigkeit der Soundscape-Bewertungen belegt werden: Musik und Sprache werden als angenehmer bewertet, technische Geräusche und Anlagenrauschen hingegen als unangenehmer. Hoher Ereignisreichtum hängt mit
menschgemachten Geräuschen (z. B. Poltern, Stuhlrücken) zusammen. Diese Ergebnisse stellen eine wichtige Komponente in der umfassenden Beschreibung komplexer Geräuschumgebungen dar und ihnen kommt in Zeiten der SARS-CoV-2-Pandemie, in denen Wohnraum vermehrt heterogen genutzt werden muss, eine erhöhte Bedeutung zu.
Over the past decades, research in room acoustics has established several derivative measures of an impulse response, some of which are incorporated in the ISO 3382 standards. These parameters intend to represent perceptual qualities, but were developed without a consistent modeling of room acoustical perception. More recent research proposed comprehensive inventories of room acoustic perception that are purely based on evaluations by human subjects, such as the Room Acoustical Quality Index (RAQI). In this work RA-QI scores acquired for 70 room impulse responses were predicted from room acoustical parameters. Except for Reverberance, the prediction of RAQI factors performed rather poor. In most cases, the sound source had a greater impact on RAQI scores. All analyses are published in an online tool, where users can upload omnidirectional and binaural impulse responses, and instantly obtain and visualize several physical descriptors, as well as predicted RAQI scores for three different sound sources. So far, acceptable prediction accuracy is achieved for Reverberance, Strength, Irregular Decay, Clarity and Intimacy. Larger data sets of evaluated impulse responses are required to improve the model performance and enable reliable predictions of room acoustical quality. Therefore, the administration of RAQI evaluations within the website is currently being developed.
The soundscape approach highlights the role of situational factors in sound evaluations; however, only a few studies have applied a multi‐domain approach including sound‐related, person‐related, and time‐varying situational variables. Therefore, we conducted a study based on the Experience Sampling Method to measure the relative contribution of a broad range of potentially relevant acoustic and non‐auditory variables in predicting indoor soundscape evaluations. Here we present the comprehensive dataset for which 105 participants reported temporally (rather) stable trait variables such as noise sensitivity, trait affect, and quality of life. They rated 6.594 situations regarding the soundscape standard dimensions, perceived loudness, and the saliency of its sound components and evaluated situational variables such as state affect, perceived control, activity, and location. To complement these subject‐centered data, we additionally crowdsourced object‐centered data by having participants make binaural measurements of each indoor soundscape at their homes using a low‐(self‐)noise recorder. These recordings were used to compute (psycho‐)acoustical indices such as the energetically averaged loudness level, the A‐weighted energetically averaged equivalent continuous sound pressure level, and the A‐weighted five‐percent exceedance level. This complex hierarchical data can be used to investigate time‐varying non‐auditory influences on sound perception and to develop soundscape indicators based on the binaural recordings to predict soundscape evaluations.
Music-related decision-making encompasses a wide range of behaviours including those associated with listening choices, composition and performance, and decisions involving music education and therapy. Although research programmes in psychology and economics have contributed to an improved understanding of music-related behaviour, historically, these disciplines have been unconnected. Recently, however, researchers have begun to bridge this gap by employing tools from behavioural economics. This article contributes to the literature by providing a discussion about the benefits of using behavioural economics in music-decision research. We achieve this in two ways. First, through a systematic review, we identify the current state of the literature within four key areas of behavioural economics-heuristics and biases, social decision-making, behavioural time preferences, and dual-process theory. Second, taking findings of the literature as a starting point, we demonstrate how behavioural economics can inform future research. Based on this, we propose the Behavioural Economics of Music (BEM), an integrated research programme that aims to break new ground by stimulating interdisciplinary research in the intersection between music, psychology, and economics.
Audioinhaltsanalyse und Multilevelmodellierung zur Vorhersage der Bewertung von Indoor Soundscapes
(2022)
Room reverberation alters the spatial impression and timbre of a sound by modulating its spectral and temporal characteristics. Thus, we argue that, on a perceptual level, reverberation basically breaks down into interaural differences and spectro-temporal cues and that the separation of a perceived timbre into a sound source and a surrounding room is a purely cognitive process. To investigate the connection between the perception of reverberation cues and timbre analysis, the sensitivity for changes in reverberation was compared to timbre perception abilities. The Timbre Perception Test was used to measure the perception of the temporal envelope, spectral centroid, and spectral flux of artificial sounds. Sensitivity for changes in reverberation time was tested with a discrimination task using speech and noise with speech-alike spectral and temporal envelopes as source signals. Musical and acoustical expertise was assessed through the Goldsmiths Musical Sophistication Index and self-reports on experience with and knowledge of acoustics. There was a considerable correlation between timbre and reverberance perception ability, but timbre perception and academic experience predicted only 41% of the variance in reverberance perception. Still, perception abilities related to similar acoustical phenomena seem to be better indicators of listening skills than self-reports on acoustical or musical expertise.
Musicians and music professionals are often considered to be expert listeners for listening tests on room acoustics. However, these tests often target acoustic parameters other than those typically relevant in music such as pitch, rhythm, amplitude, or timbre. To assess the expertise in perceiving and understanding room acoustical phenomena, a listening test battery was constructed to measure the perceptual sensitivity and cognitive abilities in the identification of rooms with different reverberation times and different spectral envelopes. Performance in these tests was related to data from the Goldsmiths Musical Sophistication Index, self-reported previous experience in music recording and acoustics, and academic knowledge on acoustics. The data from 102 participants show that sensory and cognitive abilities are both correlated significantly with musical training, analytic listening skills, recording experience, and academic knowledge on acoustics, whereas general interest in and engagement with music do not show any significant correlations. The regression models, using only significantly correlated criteria of musicality and professional expertise, explain only small to moderate amounts (11%–28%) of the variance in the “room acoustic listening expertise” across the different tasks of the battery. Thus, the results suggest that the traditional criteria for selecting expert listeners in room acoustics are only weak predictors of their actual performances.
Sound and music are well-studied aspects of the quality of experience in restaurants; the role of the room acoustical conditions, their influence on the visitors’ soundscape evaluation and their impact on the overall customer satisfaction in restaurants, however, has received less scientific attention. The present field study therefore investigated whether sound pressure level, reverberation time, and soundscape pleasantness can predict factors associated with overall restaurant quality. In total, 142 persons visiting 12 restaurants in Berlin rated relevant acoustical and non-acoustical factors associated with restaurant quality. Simultaneously, the A-weighted sound pressure level (LA,eq,15) was measured, and the reverberation time in the occupied state (T20,occ) was obtained by measurements performed in the unoccupied room and a subsequent calculation of the occupied condition according to DIN 18041. Results from linear mixed-effects models revealed that both the LA,eq,15 and T20,occ had a significant influence on soundscape pleasantness and eventfulness, whereby the effect of T20,occ was meditated by the LA,eq,15. Also, the LA,eq,15 as well as soundscape pleasantness were significant predictors of overall restaurant quality. A comprehensive structural equation model including both acoustical and non-acoustical factors, however, indicates that the effect of soundscape pleasantness on overall restaurant quality is mediated by the restaurant’s atmosphere. Our results support and extend previous findings which suggest that the acoustical design of restaurants involves a trade-off between comfort and liveliness, depending on the desired character of the place.
Through constant technical progress, multi-user virtual reality is transforming towards a social activity that is no longer only used by remote users, but also in large-scale location-based experiences. We evaluate the usage of realtime-tracked avatars in co-located business-oriented applications in a "guide-user-scenario" in comparison to audio only instructions. The present study examined the effect of an avatar-guide on the user-related factors of Spatial Presence, Social Presence, User Experience and Task Load in order to propose design guidelines for co-located collaborative immersive virtual environments. Therefore, an application was developed and a user study with 40 participants was conducted in order to compare both guiding techniques of a realtime-tracked avatar guide and a non-visualised guide with otherwise constant conditions. Results reveal that the avatar-guide enhanced and stimulated communicative processes while facilitating interaction possibilities and creating a higher sense of mental immersion for users. Furthermore, the avatar-guide appeared to make the storyline more engaging and exciting while helping users adapt to the medium of virtual reality. Even though no assertion could be made concerning the Task Load factor, the avatar-guide achieved a higher subjective value on User Experience. Due to the results, avatars can be considered valuable social elements in the design of future co-located collaborative virtual environments.
Singing in different rooms: Common or individual adaptation patterns to the acoustic conditions?
(2020)
The effect of inattention and cognitive load on unpleasantness judgments of environmental sounds
(2020)
Der vierte Teil der Schriftenreihe "Trading" präsentiert eine statistische Auswertung von Kursdaten bei einer Haltefrist von einem Handelstag. Die Statistik beinhaltet Gewinnfaktor bzw. Rendite und Investitionsgrad aus Teil 1 sowie Volatilität, Chance, Risiko und Handelskosten, welche in Teil 3 eingeführt wurden. Kaufbedingungen werden wie in Teil 1 mit Hilfe des Kursverhältnisses an zwei aufeinander folgenden Handelstagen gebildet. Die Auswertung für den Markt FEBRDUSA_1 historischer Kursdaten liefert eine in sich schlüssige Beschreibung des Einflussfaktors Kaufbedingung auf die künftige Kursentwicklung. Den größten Gewinn liefert eine Mean Reversion-Strategie, bei der die Chance deutlich über dem Risiko liegt. Da die Statistik die Begrenzung eingegangener Neuinvestitionen berücksichtigt, werden Übertreibungen vermieden, die einer arithmetischen Mittelwertbildung anhaften. Die statische Auswertung für einen mit Random Walks gebildeten Markt bestätigt, dass für diesen Fall keine Abhängigkeit von der Kaufbedingung besteht.
The PSFC, or Pioneer sound field control system, is a DSP-driven hemispherical 14-loudspeaker array, installed at the University of Aizu Multimedia Center. Collocated with a large screen rear-projection stereographic display the PSFC features realtime control of virtual room characteristics and direction of two separate sound channels, smoothly steering them around a configurable soundscape. The PSFC controls an entire sound field, including sound direction, virtual distance, and simulated environment (reverb level, room size and liveness) for each source. It can also configure a dry (DSP-less) switching matrix for direct directionalization. The PSFC speaker dome is about 14 m in diameter, allowing about twenty users at once to comfortably stand or sit near its sweet spot.
In a virtual reality environment users are immersed in a scene with objects which might produce sound. The responsibility of a VR environment is to present these objects, but a system has only limited resources, including spatialization channels (mixels), MIDI/audio channels, and processing power. The sound spatialization resource manager controls sound resources and optimizes fidelity (presence) under given conditions. For that a priority scheme based on human psychophysical hearing is needed. Parameters for spatialization priorities include intensity calculated from volume and distance, orientation in the case of non-uniform radiation patterns, occluding objects, frequency spectrum (low frequencies are harder to localize), expected activity, and others. Objects which are spatially close together (depending on distance and direction) can be mixed. Sources that can not be spatialized can be treated as a single ambient sound source. Important for resource management is the resource assignment, i.e., minimizing swap operations, which makes it desirable to look-ahead and predict upcoming events in a scene. Prediction is achieved by monitoring objects’ speed and past evaluation values. Fidelity is contrasted for Zifferent kind of resource restrictions and optimal resource assignment based upon unlimited dynamic scene look-ahead. To give standard and comparable results, the VRML 2.0 specification is used as an application programmer interface. Applicability is demonstrated with a helical keyboard, a polyphonic MIDI stream driven animation including user interaction (user moves around, playing together with programmed notes). The developed sound spatialization resource manager gives improved spatialization fidelity under runtime constraints. Application programmers and virtual reality scene designers are freed from the burden of assigning and predicting the sound sources.
The PSFC, or Pioneer Sound Field Controller, is a DSP-driven hemispherical loudspeaker array, installed at the University of Aizu Multimedia Center. The PSFC features realtime manipulation of the primary components of sound spatialization for each of two audio sources located in a virtual environment, including the content (apparent direction and distance) and context (room characteristics: reverberation level, room size and liveness). In an alternate mode, it can also direct the destination of the two separate input signals across 14 loudspeakers, manipulating the direction of the virtual sound sources with no control over apparent distance other than that afforded by source loudness (including no simulated environmental reflections or reverberation). The PSFC speaker dome is about 10 m in diameter, accommodating about fifty simultaneous users, including about twenty users comfortably standing or sitting near its ``sweet spot,'' the area in which the illusions of sound spatialization are most vivid. Collocated with a large screen rear-projection stereographic display, the PSFC is intended for advanced multimedia and virtual reality applications.
Given limited computational resources available for the rendering of spatial sound imagery, we seek to determine effective means for choosing whatcomponents of the rendering will provide the most audible differences in the results. Rather than begin with an analytic approach that attempts to predict audible differences on the basis of objective parameters, we chose to begin with subjective tests of how audibly different the rendering result may be heard to be when that result includes two types of sound obstruction: reflectors and occluders. Single-channel recordings of 90 short speech sounds were made in an anechoic chamber in the presence and absence of these two types of obstructions, and as the angle of those obstructions varied over a 90 degree range. These recordings were reproduced over a single loudspeaker in that anechoic chamber, and listeners were asked to rate how confident they were that the recording of each of these 90 stimuli included an obstruction. These confidence ratings can be used as an integral component in the evaluation function used to determine which reflectors and occluders are most important for rendering.
A filtering model for efficient rendering of the spatial image of an occluded virtual sound source
(1999)
Rendering realistic spatial sound imagery for complex virtual environments must take into account the effects of obstructions such as reflectors and occluders. It is relatively well understood how to calculate the acoustical consequence that would be observed at a given observation point when an acoustically opaque object occludes a sound source. But the interference patterns generated by occluders of various geometries and orientations relative to the virtual source and receiver are computationally intense if accurate results are required. In many applications, however, it is sufficient to create a spatial image that is recognizable by the human listener as the sound of an occluded source. In the interest of improving audio rendering efficiency, a simplified filtering model was developed and its audio output submitted to psychophysical evaluation. Two perceptually salient components of occluder acoustics were identified that could be directly related to the geometry and orientation of a simple occluder. Actual occluder impulse responses measured in an anechoic chamber resembled the responses of a model incorporating only a variable duration delay line and a low-pass filter with variable cutoff frequenc
A Sound Spatialization Server for a Speaker Array as an Integrated Part of a Virtual Environment
(1998)
Spatial sound plays an important role in virtual reality environments, allowing orientation in space, giving a feeling of space, focusing the user on events in the scene, and substituting missing feedback cues (e.g., force feedback). The sound spatialization framework of the University of Aizu, which supports number of spatialization backends, has been extended to include a sound spatialization server for a multichannel loudspeaker array (Pioneer Sound Field Control System). Our goal is that the spatialization server allows easy integration into virtual environments. Modeling of distance cues, which are essential for full immersion, is discussed. Furthermore, the integration of this prototype into different applications allowed us to reveal the advantages and problems of spatial sound for virtual reality environments.
A module for soundscape monitoring and visualizing resource management processes was extended for presenting clusters, generated by a novel sound source clustering algorithm. This algorithm groups multiple sound sources together into a single representative source, considering localization errors depending on listener orientation. Localization errors are visualized for each cluster using resolution cones. Visualization is done in runtime and allows understanding and evaluation of the clustering algorithm.
Level-of-detail is a concept well-known in computer graphics to reduce the number of rendered polygons. Depending on the distance to the subject (viewer), the objects’ representation is changed. A similar concept is the clustering of sound sources for sound spatialization. Clusters can be used to hierarchically organize mixels and to optimize the use of resources, by grouping multiple sources together into a single representative ource. Such a clustering process should minimize the error of position allocation of
elements, perceived as angle and distance, and also differences between velocity relative to the sink (i.e., Doppler shift). Objects with similar direction of motion and speed (relative to sink) in the same acoustic resolution cone and with similar distance to a sink can be grouped together.
A module for soundscape monitoring and visualizing resource management processes was extended for presenting clusters, generated by a novel sound source clustering algorithm. This algorithm groups multiple sound sources together into a single representative source, considering localization errors depending on listener orientation. Localization errors are visualized for each cluster using resolution cones. Visualization is done in runtime and allows understanding and evaluation of the clustering algorithm.
Level-of-detail is a concept well-known in computer graphics to reduce the number of rendered polygons. Depending on the distance to the subject (viewer), the objects’ representation is changed. A similar concept is the clustering of sound sources for sound spatialization. Clusters can be used to hierarchically organize mixelsand to optimize the use of resources, by grouping multiple sources together into a single representative source. Such a clustering process should minimize the error of position allocation of elements, perceived as angle and distance, and also differences between velocity relative to the sink (i.e., Doppler shift). Objects with similar direction of motion and speed (relative to sink) in the same acoustic resolution cone and with similar distance to a sink can be grouped together.
Cyberspatial audio applications are distinguished from the broad range of spatial audio applications in a number of important ways that help to focus this review. Most significant is that cyberspatial audio is most often designed to be responsive to user inputs. In contrast to non-interactive auditory displays, cyberspatial auditory displays typically allow active exploration of the virtual environment in which users find themselves. Thus, at least some portion of the audio presented in a cyberspatial environment must be selected, processed, or otherwise rendered with minimum delay relative to user input. Besides the technological demands associated with realtime delivery of spatialized sound, the type and quality of auditory experiences supported are also very different from those associated with displays that support stationary sound localization.
Sound spatialization is a technology which puts sound into the three dimensional space, so that it has a perceivable direction and distance. Interactive means mutually or reciprocally active. Interaction is when one action (e.g., user moves mouse) has direct or immediate influence to other actions (e.g., processing by a computer: graphics change in size). Based on this definition an introduction to sound reproduction using DVD and virtual environments is given and illustrated by applications (e.g., virtual converts).
Distance and Room Effects Control for the PSFC, an Auditory Display using a Loudspeaker Array
(2000)
The Pioneer Sound Field Controller (PSFC), a loudspeaker array system, features realtime configuration of an entire sound field,including sound source direction, virtual distance, and context of simulated environment (room characteristics: room size and liveness)for each of two sound sources. In the PSFC system, there is no native parameter to specify the distance between the sound source and sound sink (listener) and also no function to control it directrly. This paper suggests the method to control virtual distance using basic parameters: volume, room size and liveness. The implementation of distance cue is an important aspect of 3D sounds. Virtual environments supporting room effects like reverberation not only gain realism but also provide additional information to users about surrounding space. The context switch of different aural attributes is done by using an API of the Sound Spatialization Framework. Therefore, when the sound sink move through two rooms, like a small bathroom and a large living room, the context of the sink switches and different sound is obtained.
A chatspace was developed that allows conversation with 3D sound using networked streaming in a shared virtual environment. The system provides an interface to advanced audio features, such as a "whisper function" for conveying a confided audio stream. This study explores the use of spatial audio to enhance a user's experience in multiuser virtual environments.
Distance and Room Effects Control for the PSFC, an Auditory Display using a Loudspeaker Array
(2000)
The Pioneer Sound Field Controller (PSFC), a loudspeaker array system, features realtime configuration of an entire sound field, including sound source direction, virtual distance, and context of simulated environment (room characteristics: room size and liveness) for each of two sound sources. In the PSFC system, there is no native parameter to specify the distance between the sound source and sound sink (listener) and also no function to control it directrly. This paper suggests the method to control virtual distance using basic parameters: volume, room size and liveness. The implementation of distance cue is an important aspect of 3D sounds. Virtual environments supporting room effects like reverberation not only gain realism but also provide additional information to users about surrounding space. The context switch of different aural attributes is done by using an API of the Sound Spatialization Framework. Therefore, when the sound sink move through two rooms, like a small bathroom and a large living room, the context of the sink switches and different sound is obtained.
Auditory displays with the ability to dynamically spatialize virtual sound sources under real-time conditions enable advanced applications for art and music. A listener can be deeply immersed while interacting and participating in the experience. We review some of those applications while focusing on the Helical Keyboard project and discussing the required technology. Inspired by the cyclical nature of octaves and helical structure of a scale, a model of a piano-style keyboard was prepared, which was then geometrically warped into a helicoidal configuration, one octave/revolution, pitch mapped to height and chroma. It can be driven by MIDI events, real-time or sequenced, which stream is both synthesized and spatialized by a spatial sound display. The sound of the respective notes is spatialized with respect to sinks, avatars of the human user, by default in the tube of the helix. Alternative coloring schemes can be applied, including a color map compatible with chromastereoptic eyewear. The graphical display animates polygons, interpolating between the notes of a chord across the tube of the helix. Recognition of simple chords allows directionalization of all the notes of a major triad from the position of its musical root. The system is designed to allow, for instance, separate audition of harmony and melody, commonly played by the left and right hands, respectively, on a normal keyboard. Perhaps the most exotic feature of the interface is the ability to fork oneÃs presence, replicating subject instead of object by installing multiple sinks at arbitrary places around a virtual scene so that, for example, harmony and melody can be separately spatialized, using two heads to normalize the octave; such a technique effectively doubles the helix from the perspective of a single listener. Rather than a symmetric arrangement of the individual helices, they are perceptually superimposed in-phase, co-extensively, so that corresponding notes in different registers are at the same azimuth.
Virtual environment walkthrough applications are generally enhanced by a user’s interactions within a simulated architectural space, but the enhancement that stems from changes in spatial sound that are coupled with a user’s behavior are particularly important, especially within regard to creating a sense of place. When accompanied by stereoscopic image synthesis, spatial sound can immerse the user in a high-realism virtual copy of the real world. An advanced virtual environment that allow users to change realtime rendering features with a few manipulations has been shown to enable switching between different versions of a modeled space while maintaining sensory immersion. This paper reports on an experimental project in which an architectural model is being integrated into such an interactive virtual environment. The focus is on the spatial sound design for supporting interaction, including demonstrations of both the possibilities and limitations of such applications in presenting and promoting architectural designs, as well as in three-dimensional sketching.
With the virtual environment developed here, the characteristic
sound radiation patterns of musical instruments can be experienced
in real-time. The user may freely move around a musical instrument, thereby receiving acoustic and visual feedback in real-time. The perception of auditory and visual effects is intensified by the combination of acoustic and visual elements, as well as the option of user interaction. The simulation of characteristic sound radiation patterns is based on interpolating the intensities of a multichannel recording and offers a near-natural mapping of the sound radiation patterns. Additionally, a simple filter has been developed, enabling the qualitative simulation of an instrument’s characteristic sound radiation patterns to be easily implemented within real-time 3D applications. Both methods of simulating sound radiation patterns have been evaluated for a saxophone with respect to their functionality and validity by means of spectral analysis and an auditory experiment.
Using spatial audio successfully for augmented reality (AR) applications is a challenge, but is awarded with an improved user experience. Thus, we have extended the AR/VR framework \sc Morgan with spatial audio to improve users orientation in an AR application. In this paper, we investigate the users’ capability to localize and memorize spatial sounds (registered with virtual or real objects). We discuss two scenarios. In the first scenario, the user localizes only sound sources and in the second scenario the user memorizes the location of audio-visual objects. Our results reflect spatial audio performance within the application domain and show which technology pitfalls still exist. Finally, we provide design recommendations for spatial audio AR environments.
Virtual TV studios use actor tracking systems for resolving the occlusion of computer graphics and studio camera image. The actor tracking delivers the distance between actor and studio camera. We deploy a photonic mixing device, which captures a depth map and a luminance image at low resolution. The renderer engines gets one depth value per actor using the OSC protocol. We describe the actor recognition algorithm based on the luminance image and the depth value calculation. We discuss technical issues like noise and calibration.
The Sound Spatialization Framework is a C++ toolkit and development environment for providing advanced sound spatialization for virtual reality and multimedia applications. The Sound Spatialization Framework provides many powerful display and user-interface features not found in other sound spatialization software packages. It provides facilities that go beyond simple sound source spatialization: visualization and editing of the soundscape, multiple sinks, clustering of sound sources, monitoring and controlling resource management, support for various spatialization backends, and classes for MIDI animation and handling.
Keywords:
sound spatialization, resource management, virtual environments, spatial sound authoring, user interface design, human-machine interfaces
This paper presents an approach to integrate non-visual user feedback in today's virtual tv studio productions. Since recent studies showed that systems providing vibro-tactile feedback are not sufficient for replacing the common visual feedback, we developed an audio-based solution using an in ear headphone system, enabling a talent to move, avoid and point to virtual objects in a blue or green box. The system consists of an optical head tracking system, a wireless in ear monitor system and a workstation, which performs all application and audio processing. Using head related transfer functions, the talent gets directional and distance cues. Past research showed, that generating reflections of the sounds and simulating the acoustics of the virtual room helps the listener to conceive the acoustical feedback, we included this technique as well. In a user study with 15 participants the performance of the system was evaluated.
Ziel des Projektes ist die Entwicklung einer virtuellen Umgebung, die das charakteristische Klangabstrahlverhalten eines Musikinstruments in Echtzeit erfahrbar macht. Es wird eine virtuelle Umgebung geschaffen, in der sich der Benutzer frei um ein Musikinstrument bewegen kann und in Echtzeit ein akustisches und visuelles Feedback erhält. Durch die Verbindung der auditiven und visuellen Elemente und die Möglichkeit der Interaktion, wird das Erleben und die Wahrnehmung der Effekte intensiviert. Die Simulation des charakteristischen Klangabstrahlverhaltens erfolgt nicht durch eine rechenaufwändige Klangsynthese wie z.B. Physical Modeling, sondern basiert auf der Lautstärkeinterpolation einer Mehrkanalaufnahme. Die Verwendung der realen Aufnahmen ermöglicht eine annähernd naturgetreue Abbildung des Klangabstrahlverhaltens und ist, im Gegensatz zu rechenaufwändigen Klangsyntheseverfahren, echtzeitfähig. Zusätzlich wurde ein einfacher Filter entwickelt, der das charakteristische Klangabstrahlverhalten des Instruments eher qualitativ simuliert und sich problemlos in Echtzeit 3D-Anwendungen implementieren lässt. Die beiden entwickelten Methoden zur Simulation des Klangabstrahlverhaltens wurden mittels Spektralanalyse und anhand eines durchgeführten Hörtests auf ihre Funktionalität und ihre Gültigkeit überprüft.
Broader use of virtual reality environments and sophisticated animations spawn a need for spatial sound. Until now, spatial sound design has been based very much on experience and trial and error. Most effects are hand-crafted, because good design tools for spatial sound do not exist. This paper discusses spatial sound authoring and its applications, including shared virtual reality environments based on VRML. New utilities introduced by this research are an inspector for sound sources, an interactive resource manager, and a visual soundscape manipulator. The tools are part of a sound spatialization framework and allow a designer/author of multimedia content to monitor and debug sound events. Resource constraints like limited sound spatialization channels can also be simulated.
The Sound Spatialization Framework is a C++ toolkit and development environment for providing advanced sound spatialization for virtual reality and multimedia applications. The Sound Spatialization Framework provides many powerful display and user-interface features not found in other sound spatialization software packages. It provides facilities that go beyond simple sound source spatialization: visualization and editing of the soundscape, multiple sinks, clustering of sound sources, monitoring and controlling resource management, support for various spatialization backends, and classes for MIDI animation and handling.