Virtual Reality: The Challenges of Immersive Sound

Virtual Reality Soundscape VR Binaural Recorder

There's been a growing trend. You've probably noticed it. Articles have been popping up across social media and news platforms. Seemingly announcements are made every few days from industry leads such as Sony, Samsung, HTC, Microsoft... need we go on? It's all for one looming monolith on the horizon: the black box called 'Virtual Reality', or VR for short.

We've noticed its rise in notoriety here at Voice Squad. Over the past year there has been a great increase in requests for voiceover work for virtual reality, particularly in London. This isn't just for the founding purpose of VR, gaming. It extends much further to corporate and commercial pieces - even virtual therapy! Corporate pitches, shopping market promotions - you name it, and a production company is probably finding a way to link it to this growing industry. After all, that's its aim: to create an immersive virtual space in a physical world.

The possibilities are rather endless.

The Difficulties of Virtual Reality

Virtual reality isn't all smooth sailing and wonderment. Sound plays out differently in the world of virtual reality. By its own definition, VR needs to create a realistic landscape for the consumer – which includes the landscape of sound. It's about making the unbelievable believable. The reality of how we hear compared to the way sound is created in traditional media is drastically different. VR sound designers have had to combat and create ways for the client to believe what they hear in these created settings.

How We Hear

Our minds hear binaurally, in other words through two hearing sources (our ears) which send little messages to our mind to create a 360 degree soundscape. However traditional craft has seen the use of mono and stereo recording to set the soundscape in one place. Only one tiny slice of that 360 degree spectrum we experience. In order to record binaural sound, some very crafty tradespeople have designed nifty set ups. They look similar to a dummy head, though sometimes are as streamline as a stand, and come with earlobes affixed to their surfaces. Within these earlobes are the all-important microphones. In using this setup, the recording mimics our natural way of hearing the world around us.

How Does It Translate Into Virtual Reality?

A similar challenge was combated by the creators of Robinson: The Journeya Playstation VR game released last week. It's about as unrealistic as an expedition can get: dinosaurs, robots and outer space - oh my! Yet through clever sound design, it doesn't matter where floating sidekick HIGS is, you'll be able to hear our Glen McCready's calming (or worried) tones and know exactly where he's placed; whether that's in front of you, behind you, or far off in the distance. Ah, yes, the greatest challenge of binaural recordings: to ensure the situation of the voice matches the situation of the image.

However, the binaural recorder mentioned above would not be able to completely replicate this trick, as it still records a fairly one-directional scope of the 3D space around us. Virtual reality's needs the sound to mould itself around the moment-of-choice by the player. For example, if they choose to look at HIGS on their left, rather than the dinosaur looming in front of them, then the sound needs to naturally follow that audience-driven decision, whereas a traditional binaural recorder would still follow the looming dinosaur in front.

As such Crytek, and indeed all makers of virtual reality experiences, have a much harder task when it comes to creating a soundscape. Particularly one which follows the audience's autonomous choice. One engineer, Chris Milk, created a 'mutant' omni-binaural recorder (pictured above) with eight ears to attempt such a feat! Now, add to that the sheer quantity of audio-engineering the final product needs to piece it all together... Well, let's just say we're happy we just need to provide the voices.

There's one thing that's for sure when it comes to virtual reality, there's much more than just seeing and believing. You have to hear it too.


Robinson: The Journey, featuring Glen McCready as HIGS: