# A/B Testing Experiences **URL:** \ **Category:** General Headphone Discussion\ **Created:** [August 11, 2020, 12:21pm UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877 "2020-08-11T12:21:53Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Panzer\_Applehusky](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.headphones.com/panzer_applehusky/32/16189_2.png) [@Panzer\_Applehusky](https://forum.headphones.com/u/Panzer_Applehusky)\ **Post date:** [August 11, 2020, 12:21pm UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877/1 "2020-08-11T12:21:53Z")
After sending my Schiit Magni 3+ in for warranty service a few weeks ago, I decided to get a cheaper amp to power my HD6XX in the meantime. I ended up going with a FiiO A3 that I found on Craigslist for ~$30. Fast forward some time and I finally received my Schiit back from service, good as new. But this left me with a decision to make. Do I keep both, or sell one? Do I really need both? If I do just keep one, which one would it be? The only reasonable answer was SCIENCE! So I ordered an A/B switch off of Amazon and had my brother help me with a blind A/B test between the two amps with my trusty HD6XX. We volume matched both amps to a somewhat louder than normal volume using a constant test tone. I put the headphones on, closed my eyes and he started playing AC/DC. Once he clicked the switch several times in quick succession to ‘shuffle’ which amp I was listening to, he started switching from one to the other in random intervals. Once I had gained enough confidence, I told him to stop, and indicated which amp I thought I was listening to. During the tests, I was never very confident in my decision, but to my surprise, I was able to pick the correct one every single time. To me the only difference seemed to be the clarity in the upper frequencies. However, using my brother’s HD8 DJ’s, there was absolutely no difference. The results were random. I figured I’d share my experience and see what everyone else’s experiences are when it comes to direct A/B testing DACs and amps. If you guys have any tips or methods to make this more effective, or if you have any interesting anecdotes, I’d like to hear. ---
**Author:** ![Pancake](https://avatars.discourse-cdn.com/v4/letter/p/ecd19e/32.png) [@Pancake](https://forum.headphones.com/u/Pancake)\ **Post date:** [August 11, 2020, 2:18pm UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877/2 "2020-08-11T14:18:34Z")
Curious which A/B switch you used? ---
**Author:** ![Panzer\_Applehusky](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.headphones.com/panzer_applehusky/32/16189_2.png) [@Panzer\_Applehusky](https://forum.headphones.com/u/Panzer_Applehusky)\ **Post date:** [August 11, 2020, 2:43pm UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877/3 "2020-08-11T14:43:45Z")
I used this one [https://www.amazon.com/KNACRO-Switching-Switch-Select-Output/dp/B07V5LS96J/ref=sr\_1\_28?dchild=1&keywords=headphone+AB+switch&qid=1597157001&s=electronics&sr=1-28](https://www.amazon.com/KNACRO-Switching-Switch-Select-Output/dp/B07V5LS96J/ref=sr_1_28?tag=headphoneforum-20) ---
**Author:** ![bpcarb](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.headphones.com/bpcarb/32/16496_2.png) [@bpcarb](https://forum.headphones.com/u/bpcarb)\ **Post date:** [August 11, 2020, 3:47pm UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877/4 "2020-08-11T15:47:01Z")
Good stuff, especially blind and volume matching. You might want to try introducing triangulation, both to test/hone your ability to identify and to increase reliability of the results. For example, have him shuffle but not actually switch. And then have him play 3 samples, two of A and 1 of B, and you have to properly identify. Gain settings sometimes sound different also. For science! ---
**Author:** ![Panzer\_Applehusky](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.headphones.com/panzer_applehusky/32/16189_2.png) [@Panzer\_Applehusky](https://forum.headphones.com/u/Panzer_Applehusky)\ **Post date:** [August 11, 2020, 3:54pm UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877/5 "2020-08-11T15:54:53Z")
I think that sounds like a great idea! I just wrote up a post for an automated solution to this. To my knowledge, apart from having a friend flip the switch, there’s no way to perform a good blind test. ---
**Author:** ![bpcarb](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.headphones.com/bpcarb/32/16496_2.png) [@bpcarb](https://forum.headphones.com/u/bpcarb)\ **Post date:** [August 11, 2020, 4:02pm UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877/6 "2020-08-11T16:02:52Z")
I’ll check out your post. For headphones, it’s tough to do blind comparisons independently (as far as I know). It’s easier for blind coffee/tea/libations tasting. If these comparisons interest you, you might want to read our DNA Starlett tube amp thread where I discuss my A/B comparisons (granted, not blind) of the various headphone outputs (TCR vs XLR, gain, impedance). ---
**Author:** ![Panzer\_Applehusky](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.headphones.com/panzer_applehusky/32/16189_2.png) [@Panzer\_Applehusky](https://forum.headphones.com/u/Panzer_Applehusky)\ **Post date:** [August 11, 2020, 4:07pm UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877/7 "2020-08-11T16:07:01Z")
That sounds like a good read. Do you have a link? ---
**Author:** ![bpcarb](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.headphones.com/bpcarb/32/16496_2.png) [@bpcarb](https://forum.headphones.com/u/bpcarb)\ **Post date:** [August 11, 2020, 4:17pm UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877/8 "2020-08-11T16:17:38Z")
Here’s the most recent one and I believe you can trace precedents, or scroll up. > [@DNA Starlett - An Attempted Review](https://forum.headphones.com/t/dna-starlett-an-attempted-review/6928/270): > > This is an update of my output comparison. The biggest difference is TCR low vs high impedance. I prefer how low impedance is noticeably less constricted. I noticed no material difference between XLR SE and balanced [EDIT: upon further listening, XLR SE has more detail in the treble region in a favorable way]. TCR low impedance, high vs low gain is the most interesting comparison. Low gain is more dense but laid back, whereas high gain has more air and a more forward presentation. I’ve pivote… ---
**Author:** ![Tchoupitoulas](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.headphones.com/tchoupitoulas/32/37432_2.png) [@Tchoupitoulas](https://forum.headphones.com/u/Tchoupitoulas)\ **Post date:** [August 11, 2020, 6:04pm UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877/9 "2020-08-11T18:04:01Z")
> [@Panzer\_Applehusky](#): > > see what everyone else’s experiences are when it comes to direct A/B testing I recommend reading the articles in the “Big Sound 2015” on Innerfidelity. Tyll Hertsen’s contributions, including the conclusions for which I’m providing the following link, are fascinating, as are the other participants’ reflections; in fact, the whole thing is well worth reading: [Big Sound 2015 Wrap: What I Learned | InnerFidelity](https://web.archive.org/web/20170331225330/http://www.innerfidelity.com/content/big-sound-2015-wrap-what-i-learned) (note you need to use the internet wayback machine to access the articles now that IF is defunct). Besides the valuable things the contributors have to say about blind tests, the articles also offer fascinating insights into what constituted the summit of headphone listening half a decade ago. It’s interesting to consider how things have changed since then (or not?). ---
**Author:** ![cpp](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.headphones.com/cpp/32/14979_2.png) [@cpp](https://forum.headphones.com/u/cpp)\ **Post date:** [August 11, 2020, 10:51pm UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877/10 "2020-08-11T22:51:06Z")
Thanks for posting that link. It holds a lot of truth. ---
**Author:** ![AudioTool](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.headphones.com/audiotool/32/14289_2.png) [@AudioTool](https://forum.headphones.com/u/AudioTool)\ **Post date:** [August 12, 2020, 12:06am UTC](https://forum.headphones.com/t/a-b-testing-experiences/7877/11 "2020-08-12T00:06:08Z")
The comments on the article are almost as interesting as the article itself. I hope everybody clicked through to the post by Mike Moffat of Schiit: [https://www.head-fi.org/threads/schiit-happened-the-story-of-the-worlds-most-improbable-start-up.701900/page-516#post-11921090](https://www.head-fi.org/threads/schiit-happened-the-story-of-the-worlds-most-improbable-start-up.701900/page-516#post-11921090)