Refine
Year of publication
Document Type
- Article (12)
- Conference Proceeding (12)
- Researchdata (1)
Language
- English (25) (remove)
Is part of the Bibliography
- yes (25) (remove)
Keywords
- VSVR (11)
- FHD (3)
- DOAJ (2)
- spatial audio (2)
- 3D audio (1)
- Accelerometer (1)
- Avatars (1)
- Co-located Collaborations (1)
- Dataset (1)
- Embodied Music Cognition (1)
Department/institution
- Sound and Vibration Engineering (25) (remove)
Auditory displays with the ability to dynamically spatialize virtual sound sources under real-time conditions enable advanced applications for art and music. A listener can be deeply immersed while interacting and participating in the experience. We review some of those applications while focusing on the Helical Keyboard project and discussing the required technology. Inspired by the cyclical nature of octaves and helical structure of a scale, a model of a piano-style keyboard was prepared, which was then geometrically warped into a helicoidal configuration, one octave/revolution, pitch mapped to height and chroma. It can be driven by MIDI events, real-time or sequenced, which stream is both synthesized and spatialized by a spatial sound display. The sound of the respective notes is spatialized with respect to sinks, avatars of the human user, by default in the tube of the helix. Alternative coloring schemes can be applied, including a color map compatible with chromastereoptic eyewear. The graphical display animates polygons, interpolating between the notes of a chord across the tube of the helix. Recognition of simple chords allows directionalization of all the notes of a major triad from the position of its musical root. The system is designed to allow, for instance, separate audition of harmony and melody, commonly played by the left and right hands, respectively, on a normal keyboard. Perhaps the most exotic feature of the interface is the ability to fork oneÃs presence, replicating subject instead of object by installing multiple sinks at arbitrary places around a virtual scene so that, for example, harmony and melody can be separately spatialized, using two heads to normalize the octave; such a technique effectively doubles the helix from the perspective of a single listener. Rather than a symmetric arrangement of the individual helices, they are perceptually superimposed in-phase, co-extensively, so that corresponding notes in different registers are at the same azimuth.
The effect of inattention and cognitive load on unpleasantness judgments of environmental sounds
(2020)
Music-related decision-making encompasses a wide range of behaviours including those associated with listening choices, composition and performance, and decisions involving music education and therapy. Although research programmes in psychology and economics have contributed to an improved understanding of music-related behaviour, historically, these disciplines have been unconnected. Recently, however, researchers have begun to bridge this gap by employing tools from behavioural economics. This article contributes to the literature by providing a discussion about the benefits of using behavioural economics in music-decision research. We achieve this in two ways. First, through a systematic review, we identify the current state of the literature within four key areas of behavioural economics-heuristics and biases, social decision-making, behavioural time preferences, and dual-process theory. Second, taking findings of the literature as a starting point, we demonstrate how behavioural economics can inform future research. Based on this, we propose the Behavioural Economics of Music (BEM), an integrated research programme that aims to break new ground by stimulating interdisciplinary research in the intersection between music, psychology, and economics.
Using spatial audio successfully for augmented reality (AR) applications is a challenge, but is awarded with an improved user experience. Thus, we have extended the AR/VR framework \sc Morgan with spatial audio to improve users orientation in an AR application. In this paper, we investigate the users’ capability to localize and memorize spatial sounds (registered with virtual or real objects). We discuss two scenarios. In the first scenario, the user localizes only sound sources and in the second scenario the user memorizes the location of audio-visual objects. Our results reflect spatial audio performance within the application domain and show which technology pitfalls still exist. Finally, we provide design recommendations for spatial audio AR environments.
With the virtual environment developed here, the characteristic
sound radiation patterns of musical instruments can be experienced
in real-time. The user may freely move around a musical instrument, thereby receiving acoustic and visual feedback in real-time. The perception of auditory and visual effects is intensified by the combination of acoustic and visual elements, as well as the option of user interaction. The simulation of characteristic sound radiation patterns is based on interpolating the intensities of a multichannel recording and offers a near-natural mapping of the sound radiation patterns. Additionally, a simple filter has been developed, enabling the qualitative simulation of an instrument’s characteristic sound radiation patterns to be easily implemented within real-time 3D applications. Both methods of simulating sound radiation patterns have been evaluated for a saxophone with respect to their functionality and validity by means of spectral analysis and an auditory experiment.
Virtual TV studios use actor tracking systems for resolving the occlusion of computer graphics and studio camera image. The actor tracking delivers the distance between actor and studio camera. We deploy a photonic mixing device, which captures a depth map and a luminance image at low resolution. The renderer engines gets one depth value per actor using the OSC protocol. We describe the actor recognition algorithm based on the luminance image and the depth value calculation. We discuss technical issues like noise and calibration.