The Evolution of Spatial Music
What is the relationship between visual, music and architecture, or space? How do we explore narratives and interfaces by non-visual means of time and sound, beyond the general concept of visualization and materialization? How do we implement space as a means for acoustic narrative, which helps people, especially visually impaired person to participate decision making and production?
Driven by such questions, several innovations on spatial DAW system beginning in 2019 explored the idea of spatial music structure(virtual) and instrument(physical). The innovation process provides a new answer of nowadays to the question of space-time as well as architecture-music.
Spatial Live Set
Inspired by structure of Live Set Performance, as an allegory to urban space structure. The system enables performance through gestures.
Space is divided into different regions, which is similar to a spatialization of a “Live Set” structure, functions as spatially distributed audio groups. Each group has its own attributes(loop, speed, quantization clocks, volume, audio effects and reverb sizes). While the represented user location switching from one region to another, different audio groups are triggered on clock, like a musical system. Meanwhile, the height of user/hand defines the parameter of audio effect, which is defined as frequency value of a low-pass filter in the set of GAP+.
Spatial Modulation
Distance-related Audio Effect Modulation
Continuing the idea of Spatial Live Set, this system enables explores accessibility and interactive music with Distance-related Audio Effect Modulation in AR/XR. It informs user audio cues, and reshape audio narrative with instructed movements of user.
The distance relationship is reflected as audio cues of audio effect modulation, by modulating parameters such as frequency of filter, grain size of granular synths or pitch of sound. As a progressive musical structure used in electronic musics, the modulation outcome produces musical feedback and affective effects. By informing users their relationship to targets, users move in space with audio – audio guides movement, and movement reshapes audio.
Distance-Related Audio Effect Modulation
From a HRTF spatialization and Spatial Modulation standpoint, the system provide multi-branch and recursive musical narrative enabled by audience. With speaker arrays of installations, this system provide advanced human-centered experience in the sound-field.
Each circular structure contains 1-3 target objects, that are spatialized by binaural panning. Distance to each target is remapped to the macro parameters of each audio object’s DSP, producing weaved parallel progressions and musical narratives.
FUTURE WORK
Coming soon……