Using TrackIR to improve 3D headphone audio

[hurleybird]

That's not what the thread is about. It's not about being able to use the TrackIR to create binaural effects with headphones. For example, you could emulate a pair of speakers or even a stage when listening to audio files. Moving your head would then change the location of the stage in proportion to your head, and the sound going to each ear would change accordingly. The more channels an audio source has the more immersive an experience you can make.

[Seth Steiling]

Hi hurleybird. The original discussion predates my time at NP by quite a bit, so I can't comment on how things panned out originally. But, our general approach is to keep what we offer to devs pretty foundational--your head is located x, y, z and it's oriented a, b, c. So, if they want to utilize that information for view control (which is almost universally the case), they're welcome to. If they also want to binaurally adjust the in-game sound effects (which would be awesome, and certainly both add to immersion and maintain the integrity of your situational awareness to more accurately reflect your position/orientation), that's all the better. Our SDK doesn't necessitate one, the other, or both.

So, your best bet would be to recommend adding this level of support for TrackIR directly to the game's devs. The programmatic burden to make it happen would be on them in the end. Having said that, I do think it's a sweet idea, and I'll probably float it to a couple of devs myself, just to see if they bite. As far as us developing a driver-level mod to accommodate this in other forms of entertainment--movies for instance--I don't see room for that on our roadmap any time soon. As cool as it could be if it works, we have our dev resources allocated quite a ways out now.

[hurleybird]

Well, assuming the view port moves when you move your head one would think most games would already adjust positional audio based on that alone, and if proper headphone support was built in to the game, it should do it binaurally. As for music and games, it would be just enough to keep things in mind for future road maps. Being able to use trackIR not just as a games device, but a full blown multimedia device would sure grow the range of uses.

[Seth Steiling]

I think that the problem lies in the fact that game devs don't usually program a differentiation of viewport, aiming, and body orientation (at least on yaw). They're all tied to each other, with 2 of them being slaved to whichever they assign priority to. Usually, TrackIR will be integrated essentially as an in-house view mod, with the separation of viewport control out from the aforementioned control conglomerate. So, it's very possible that they still control positional audio with body orientation or aim. This would be a slight oversight, if it is the case, as it would be better to move it to be driven by TrackIR as well for the reasons you brought up. But, I can see something like this getting missed.

All in all, good thoughts, and a worthwhile discussion for us to have internally. I'll bump this up the chain, for discussion's sake. As I mentioned, any potential development on our part would be a long ways out, most likely.

[thomase]

Wow, this thread sat idle for 4.5 years before someone picked it up again!

It seems that the way TrackIR is implemented, small head movements are amplified to large viewport movements on screen. If the binaural audio were dependent on the viewport, I wonder if this would cause some disorientation due to the fact that the sound field "moves" more than the head position would suggest.

One of the points of this idea is the binaural effect is enhanced because the user can make slight head movements to resolve the location of a sound when there is uncertainty. I imagine that we often do this in real life. I'm not sure if this would still work if the viewport angle doesn't map 1:1 with the head angle, unless of course the binaural audio uses the non-amplified head position instead of the viewport.