In coordination with iARTA, I developed a Unity environment with procedural shape generation and a custom video-reactive shader.
VRSoMa is an immersive sonic playground where users can shape sound movement in a 3D virtual environment. “Grab” a sound object to move it around the space, place waypoints to design intricate spatial trajectories, or use gravity simulations to create complex orbital patterns. All generated sounds can be recorded for later playback on headphones or multi-channel speaker arrangements. Development is ongoing, with plans to release a non-VR version.
Built in the Unity video game engine, VRSoMa was created with the purpose of using virtual reality as a tool for designing 3D spatial audio experiences. While many tools exist for this purpose, most rely on a 2D interface to position sounds. Virtual reality presents an opportunity to work with 3D sound in a more intuitive manner, utilizing the HTC Vive’s room tracking technology to operate in a 3D generated environment.
— Support for 1st order ambisonic recording (Spatialization done with Google Resonance)
— Audio reactive shader to increase visual perception
— Recording of animation data for sound objects
— Waypoint system to automate sound trajectories
— Gravity simulations to move sound objects using Playing with Gravity
Strategies for the Creation of Spatial Audio in Electroacoustic Music
This paper discusses technical and conceptual approaches to incorporate 3D spatial movement in electroacoustic music. The Ambisonic spatial audio format attempts to recreate a full sound field (with height information) and is currently a popular choice for 3D spatialization. While tools for Ambisonics are typically designed for the 2D computer screen and keyboard/mouse, virtual reality offers new opportunities to work with spatial audio in a 3D computer generated environment. An overview of my custom virtual reality software, VRSoMa, demonstrates new possibilities for the design of 3D audio. Created in the Unity video game engine for use with the HTC Vive virtual reality system, VRSoMa utilizes the Google Resonance SDK for spatialization. The software gives users the ability to control the spatial movement of sound objects by manual positioning, a waypoint system, animation triggering, or through gravity simulations. Performances can be rendered into an Ambisonic file for use in digital audio workstations. My work Discords (2018) for 3D audio facilitates discussion of the conceptual and technical aspects of spatial audio for use in musical composition. This includes consideration of human spatial hearing, technical tools, spatial allusion/illusion, and blending virtual/real spaces. The concept of spatial gestures has been used to categorize the various uses of spatial motion within a musical composition.
Some recordings I did with the Sennheiser AMBEO VR Microphone. Rendered to binaural format (listen with headphones).
The first recording is a pair of crash symbols moved around the microphone. The second recording is from a power plant on the University of North Texas campus. I placed different objects over the capsule to filter out the power plant and highlight the objects’ resonant frequencies (mason jar, metal thermos, etc.).
A selection of samples I’ve recorded over the past few years.
Bağlama (Turkish stringed instrument) 16 mono samples (44.1kHz, 16bit). A couple of normal plucked sounds, but mostly the samples were created with an Ebow and loosening the strings.
Prepared Classical Guitar 31 stereo samples (44.1kHz, 24bit). Created using a classical guitar and Allen wrench (no damage was done).
Electric Guitar 16 mono samples (44.1kHz, 16bit). Some sounds from my Ibanez s770pb, making heavy use of the tremolo arm and Ebow.
Physical Modeled Flute 22 mono samples (44.1kHz, 16bit). Sounds are from a Max/MSP patch I built based around a physical modeled flute designed by Jon C. Nelson. *Warning* Some sounds are quite loud and harsh. I have plans to share the Max patch, however some cleaning up is needed…
Crystal Glasses 8 stereo samples (48kHz, 24bit). Performed by Jonathan Thompson in the UNT recording lab (shared with his permission).
Miscellaneous 17 stereo samples (44.1kHz, 24bit). Sounds created with some cardboard, brush, and Allen wrench.