NVIDIA VRWorks SDKs Make Stitching 360-Degree Audio, Video Easier

Virtual reality has never before been as accessible as it is today but it still hasn't hit mainstream the way smartphones have. Part of that is due to the exorbitant price tags for equipment and part is due to the lack of noteworthy VR content. The latter, in turn, is due to the lack of tools to create such content. At the GPU Technology Conference (GTC), NVIDIA, one of the biggest names in the industry, is unveiling two VRWorks SDKs to address exactly that. And, surprisingly enough, the more mature and stable of the two is VRWorks Audio.

It's easy enough to take audio for granted in VR and, really, in many games. The medium, after all, is primarily a visual one. But the illusion of immersion in a make believe world also depends on the other senses, especially audio. Fortunately, audio is somewhat easier to get right than, say, haptics.

Many 360-degree audio technologies and toolkits do accurately map the position of audio in 3D space and stop at that. Sound, however, just like light, can be affected by various factors en route to our ears. Things like refraction, reflection, and diffraction. Coincidentally, NVIDIA solved this problem using a technique very familiar to graphics engineers and creators: ray-tracing.

Ray-tracing basically traces the path that audio travels from source to destination. It calculates that path based on the layout of the scene and other environmental factors that could bend its path. All of these is calculated at real-time without the SDK knowing the lay of the land, so to speak, before hand. Unsurprisingly, that's thanks to NVIDIA's monsters of graphics cards.

Those same GPUs are the muscle behind the upcoming VRWorks 360 Video SDK. This one is able to stitch together a 360-degree video coming from eight 4K cameras. Again all in real-time. This VRWorks 360 Videos SDK is still under public beta, while the VRWorks Audio SDK is already available in a public release.