Using TrackIR to improve 3D headphone audio

thomase
Posts: 8
Joined: Thu Jun 16, 2005 5:00 am

Using TrackIR to improve 3D headphone audio

Post by thomase » Thu Jun 16, 2005 2:45 am

A little background....
Many FPS games support 3D audio APIs such as OpenAL or DirectSound3D. With a supported soundcard and a pair of headphones, the game need only specify the location and orientation of the source AND the location and orientation of the listener. The soundcard applies filtering algorithms to the sound to create the ILLUSION of a 3D sound source (ie. footsteps behind your back, a gunshot to the front right and above or back left and below, etc.)

This works because the special HRTF filters try to "change" the sound as the virtual source moves around your head in the same way that your head, body, and folds of your ears would "change" the sound in real life.

However, one major shortcoming of this approach is that the brain often depends on slight movements of the head to resolve ambiguities in the direction of a sound. For example, when we hear a faint sound that could be in front or behind, we slightly (and perhaps unconsciously) move our heads in order to figure it out. If we move our head left and the sound gets louder in the left ear, it must be behind (louder in the right ear means in front). We can SIMULATE this in an FPS game by slightly adjusting our view with the mouse, but this is not the same as a subconcious movement of the head.

I'm wondering if the TrackIR product might be able to address this problem. In theory, all you would need is some device to track the exact angle of your head in reference to a straight line extending from your head to the monitor (perhaps the Vector Expansion would be of help?). There would only be minimal software support necessary in the game. Basically, under normal circumstances, the game interprets the listener orientation to correspond directly to the center of the player's first-person-view. This is the orientation sent to the 3D audio API and the soundcard. With head tracking, all that is necessary is that the game ADD an X and Y axis angle offset to this orientation to compensate for the position of the listener's head. It is not necessary to translate head movement to view movement as this would probably be more complicated (it would also move your crosshairs).

Furthermore, if support for this could be directly integrated in the soundcard's 3D audio drivers (DirectSound3D, OpenAL, etc.) the TrackIR would INSTANTLY become backward compatible with MOST existing FPS games by providing more accurate binaural hearing simulation over headphones.

Any thoughts?

P.S. I originally posted this in the wrong forum, but I have moved it here.

thomase
Posts: 8
Joined: Thu Jun 16, 2005 5:00 am

Re: Using TrackIR to improve 3D headphone audio

Post by thomase » Thu Jun 16, 2005 3:12 am

Let me repeat that if the TrackIR could be integrated to TRANSPARENTLY (ie. without chaging the actual game software) work in conjuction with soundcards, the improvements to headphone 3D audio could be dramatic enough (not to mention applicable to most existing FPS games) that it would virtually become a requirement for anyone who wishes to be competive in online FPS games.

thomase
Posts: 8
Joined: Thu Jun 16, 2005 5:00 am

Re: Using TrackIR to improve 3D headphone audio

Post by thomase » Tue Jun 21, 2005 12:13 am

No comments?

Do people not understand what I am talking about?

Jim
NaturalPoint Employee
NaturalPoint Employee
Posts: 1395
Joined: Mon Oct 14, 2002 5:00 am
Location: Corvallis, Oregon
Contact:

Re: Using TrackIR to improve 3D headphone audio

Post by Jim » Tue Jun 21, 2005 11:36 am

Hello:

Sorry for the delay in replying, I haven't checked the boards in a few days, busy with projects!

I actually love this idea, we always need new ways to make TrackIR more of a value to our customers. We have discussed this possibility in the past, but were unsure of how much of a difference it would really make to the users. From what you describe, it would make a difference for sure. I wonder how many people play with headsets on at home, I know that at a LAN party they all wear headsets.

I will forward this post to the programmers and see what we can come up with for a tech demo, perhaps we can integrate it into our Halflife 2 demo.

Thanks for the suggestion!

PS. We have been contacted by high end audio companies in the past who want to know the head position for better sound, seems like they know what they are doing, so you must be onto something!

thomase
Posts: 8
Joined: Thu Jun 16, 2005 5:00 am

Re: Using TrackIR to improve 3D headphone audio

Post by thomase » Tue Jun 21, 2005 2:41 pm

Some validation!

Thanks.

I would guess that most online FPS gamers tend to use headsets/headphones because:

1) Headset with mic is good for team chat.
2) You can hear more details with headphones, such as the footsteps of your enemies.
3) Better for late night sessions without waking people. :-)

thomase
Posts: 8
Joined: Thu Jun 16, 2005 5:00 am

Re: Using TrackIR to improve 3D headphone audio

Post by thomase » Tue Jun 21, 2005 5:31 pm

Let me rephrase and simplify.

The problem with headphones is that the 3D sound field moves as you turn your head. For example, if you hear a sound to your right and then turn your head to the right, the sound continues to come from the right even though you are now facing it.

TrackIR could solve this problem. Furthermore, it could potentially be done without patching or modding the game itself. If it were done as a patch/mod to DirectSound3D (perhaps a wrapper shared object that hijacks API calls?), it could be yet another built-in compatibility mode for the TrackIR (primarily for FPS games with DirectSound3D support) as an alternative to taking over control of the mouse.

I'd like some comments please. Even if you think its a bad idea.

marcocom
Posts: 24
Joined: Thu Dec 23, 2004 5:00 am
Location: los angeles

Re: Using TrackIR to improve 3D headphone audio

Post by marcocom » Sat Jul 23, 2005 10:35 am

when using the trackIR to present SimRacing to my client, i fashioned the Vector reflector to the headphones. it was alot more newcomer-friendly for people to put on headphones than someone-elses hat (lice and stuff i guess). it worked just as well.

what if naturalpoint mfrd a headphone with reflectors and 3d-audio input entirely integrated into one single unit? it might sell better as a product that already has a shelf space in most retail stores.

hrm...

Jim
NaturalPoint Employee
NaturalPoint Employee
Posts: 1395
Joined: Mon Oct 14, 2002 5:00 am
Location: Corvallis, Oregon
Contact:

Re: Using TrackIR to improve 3D headphone audio

Post by Jim » Sun Jul 24, 2005 7:25 am

Hello:

Thanks for the input. We do believe that headsets are a great way to mount the Vector Clip. In the future, we will be releasing software that more readily accomidates this setup.
Does anyone like the idea of a visor, you know, like for golf or poker playing? It could be fun, and not muss peoples hair.

hurleybird
Posts: 3
Joined: Sat Nov 28, 2009 9:13 am

Re: Using TrackIR to improve 3D headphone audio

Post by hurleybird » Tue Dec 01, 2009 11:04 pm

Did anything ever come from this idea? Would love to have some kind of basic binaural implementation for games, music, even movies.

VincentG
Posts: 7728
Joined: Mon Jul 17, 2006 5:00 am
Location: Corvallis, Oregon

Re: Using TrackIR to improve 3D headphone audio

Post by VincentG » Wed Dec 02, 2009 6:41 pm

We developed the TrackClip Pro, which attaches to the leftside of a headset.

Post Reply