Intro
The most debated topic in gaming headphones is unquestionably around imaging performance. Countless reviews swear that some headphone or IEM can give a player some gaming advantage through massively increased accuracy and imaging precision. Claims such as “The Sennheiser HD 800S will give you wallhacks!” are not uncommon. And gaming companies lean into this for their marketing materials. Yet there’s the argument that the headphone or IEM itself doesn’t matter much as imaging hinges entirely on the game’s engine and player skill. For example, Counter Strike pros often play at the biggest tournaments with nothing but basic IEMs under 3M noise blockers.
So what’s the truth? The only way to know is to test it. Surprisingly, I couldn’t an easy way to objectively test this in an audio only A/B environment. So I took it into my own hands to build a website to test this: imaging-lab.xyz (formally Audio Imaging Test Lab or AITL).
How It Works
At imaging-lab.xyz, there are 4 different game modes. Sandbox, Leaderboard, Comparison, and Researcher. Creating an account and logging in is recommended but not required to get started. Game data will save locally to cache first.
For the purpose of imaging testing, the most important is the A/B Test in the Comparison mode where you can input two different headphones that you want to compare the imaging ability of.
- Select the headphones, choose the stimulus (by default it’s the AK-47 gunshot sound from CS2), and choose the number of trials per side - the more the better.
- Wear your first headphone and you’ll be put in a completely pitch black (blind) room. A sound will play randomly around you - aim and click where you think it comes from. Do this until you finish the number of trials.
- Then switch to your second headphone and run the same set of tests again - every sound is randomized so no repeats.
- After finishing the trials, you’ll be sent to a screen that displays your results for each trial. To prevent cheesing, every trial calculates BOTH accuracy and speed. For any attempts that take more than 500 ms to click, a ramping penalty to the accuracy score will be applied. In other words, players can’t take their time to “find” where the sound is coming from. They have to react to it.
- All together, this calculates a T-test to determine if one headphone or IEM gives a statistically significant advantage in imaging compared to another. We’re using the standard p < 0.05 as the statistical significance threshold.
Essentially, this is a community science experiment. The null hypothesis is that headphones do not provide a difference in gaming imaging ability. In fact, I expect actual differences to be so rare that I’ve set up a page specifically to track when a difference is found. I’ll be manually verifying these as a second layer to prevent cheating for now, but maybe I’ll be proven wrong by players and there will be plenty that are legitimate A/B test winners and I’ll need to rethink this set-up. Post your results in this thread if you want to demonstrate your success!
Other Game Modes
While I originally built this app for the Comparison A/B test mode, I realized there was greater potential for this game to help gamers with their imaging ability.
Sandbox: This is a free-play mode where you can freely adjust all sorts of parameters in the side bar such as the height and width of the test environment or even upload custom sound files for practice. Seriously, check out the sidebar. There’s a TON of customization ability to be done. While it’s not a true 1:1 of a real game environment, it’s a good start to familiarize yourself and gain experience in audio spatialization.
Leaderboard: Gamers are competitive, right? So here’s an actual leaderboard for people to compete against each other in with different levels of difficulty. There’s even a link sharing system where you can challenge your friends against the session you just played!
Researcher: Based on some feedback I got from an actual PhD student studying spatial audio (thanks Nils!), I set up an even more restrictive test environment.
Your History and Community Hub
As mentioned, you can either create an account or play as a guest. Either way, your results are saved locally first and to the cloud if you created an account and upload your saved data permanently.
There’s also a Leaderboard progression tracker to measure your improvement over time on each of the difficulties.
Lastly, there are a number of community items to explore such as the A/B test winners, viewing the Leaderboards, and the Audio 101 tab to learn more about headphones and see reviews.
Closing
Let me know what you guys think! I’m very curious to see if people are able to consistently show a difference between two pieces of gear or how this site may be used to help improve someone’s gaming imaging. Over time, I hope to add more features as I see how players are using it and if there are any requests. I’ll be using this thread as place to post game updates and take feedback. But for now, I think this site is at a starting point for those serious about understanding gaming imaging.
The two most important goals for me in creating this is that: 1) People have a way to test for themselves if they can hear a difference in imaging between two pieces of gear; and 2) This can be used to hold reviewers and marketing pages to some level of honesty. If someone says they can hear a difference, they’ve gotta be able to show it here.
Don’t forget to post your results here if you think you got a statistically significant result (without cheating)!