In coordination with iARTA, I developed a Unity environment with procedural shape generation and a custom video-reactive shader.
VRSoMa is an immersive sonic playground where users can shape sound movement in a 3D virtual environment. “Grab” a sound object to move it around the space, place waypoints to design intricate spatial trajectories, or use gravity simulations to create complex orbital patterns. All generated sounds can be recorded for later playback on headphones or multi-channel speaker arrangements. Development is ongoing, with plans to release a non-VR version.
Built in the Unity video game engine, VRSoMa was created with the purpose of using virtual reality as a tool for designing 3D spatial audio experiences. While many tools exist for this purpose, most rely on a 2D interface to position sounds. Virtual reality presents an opportunity to work with 3D sound in a more intuitive manner, utilizing the HTC Vive’s room tracking technology to operate in a 3D generated environment.
— Support for 1st order ambisonic recording (Spatialization done with Google Resonance)
— Audio reactive shader to increase visual perception
— Recording of animation data for sound objects
— Waypoint system to automate sound trajectories
— Gravity simulations to move sound objects using Playing with Gravity