Hi,
Does anyone here compared Benchmark DAC 3 with RME ADI-2 DAC fs? I only have read objective measurement differences about never read subjective reviews about it.
Thank you for sharing.
Whill
Is it normal that you can hear a noise floor in Hi-Power mode when using sensitive headphones such as AD2000?
My Teak remain quiet all the way up to max volume (no sounds playing).
1 Like
I still enjoy my Orchid.
I was involved in some blind testing of 20 DACS with my local audiophile group and the RME ADI-2 finished in the top five, the four other DACS were double and quadruple its price. Fifty people were in the listening group so it was not just one person’s opinion. 
8 Likes
hey Paladin,
as RME fan it is nice to read about this - not that I would need something “proved”
we like what we like and can afford LoL
was this recently (with the 4493 version) and was there maybe also Schiit Bifrost 2 involved?
you guys were listening to a 2.0 setup or headphones?
2 Likes
It was maybe a year and a half ago and the Bifrost 2 was not out yet. The Gungnir and Yggy were in the mix as were many R-2R dacs. We did not test anything above the $7,000 range because club members donated their gear and we did not want someone reading that their $10,000 DAC finished in the middle of the pack. Schiit sent me a Bifrost 2 when they hit and I want to say that was roughly a year ago?
We did not use headphones at this time, it was a speakers. We will be using headphones with four tube amps I designed in another test and HD800;s. We use 25 criteria on a four point system and a minimum of 50 people who are performing artists, audio engineers, EE’s etc. The DAC test ran an entire weekend and it took roughly 8 hours per person.
7 Likes
thank you Paladin!
this sounds like a cool weekend with other pro´s and enthusiasts!
let´s hope we get back to normal soon, for these kind of events and meetups!
3 Likes
Any results that you could share like the top 5
2 Likes
Like the full list of 20, with exact scores? 
3 Likes
What version TARDIS was used?
4 Likes
I would most likely offend some folks if I listed the 20, I will say though that in the top five were the Aqua La Voce S3, Benchmark DAC3, Chord Hugo 2, and Auralic Vega 2.
It could be a long time before we can meet again for such tests and students are back in town now, so we will see how that goes. I live in a university town and their music department produced such people as Joshua Bell.
4 Likes
The dac shootout is certainly cool and interesting.
I remember a shootout long ago where they tested a bunch of units (can’t remember what site or if it was dacs or amps) and posted the top finishers.
It turned out they did it like a tennis tournament. They compared one against one, the winner advancing and the looser was out. Yikes!
The final results are what we tend to focus on but the methodology is important to put the results in context.
It is so much work to do these things. Gotta acknowledge and admire people who can make it happen.
Human testing procedures can be insanely complicated to obtain valid results.
With an 8 hour test session, fatigue likely set in so it’s hard to say whether the early ratings were equivalent to the later ratings. Similarly, with a knock-out model people may tend to prefer one tone profile over time (e.g., more punchy or more relaxing—TBD).
In human Learning & Memory research they employ tons of controls, reversed orders, random orders, multiple sessions, etc. to control for this stuff. It would be great to standardize the process, but awfully costly and complex.
1 Like
This is a long process because of the setup involved and at the end you are allowed to listen to any devices head to head, you only know them by number. Results are averaged and on this occasion one young lady selected the DAC;s in the same order as the entire group. That is very rare, generally the group is more accurate than any one individual within said group. There are a lot of folks from the scientific community in the group who did setup and music is specifically recorded for the tests.
1 Like
Group members donated specific pieces, I only gave a Gungnir myself because I wanted to see how it stacked up. The grueling part can be listening to the same music excerpts over and over but subjective was made as objective as possible. Many fine folks donated time and while I have a fairly complete electronics shop it does not compare with the equipment we were able to borrow just for the setup lol.
When things are safe again we will run 52 of the worlds most highly regarded 6sn7 tubes and equivalents through four identical amps I designed and this is a much easier setup. Same DAC or music source, same type amp, same type of headphones.
2 Likes
Yeah, that’s perhaps impossible.
My background is in research psychology. It’s a close cousin to biological research and often uses this type of test process. The trouble is that “brain burn-in” involves bio-psychological processes related to learning and habituation. One cannot easily unlearn, and mere exposure can have unintended side-effects (e.g., people tend to prefer or like content that they’ve previously seen; this is a fundamental behind advertising and what makes specific music popular).
If possible, split your participants into several groups and use different orders for testing. E.g., A to Z; Z to A; Unique and Random. It could also be enlightening to assess expectancy by comparing Sighted vs. Blind conditions, plus a Mislabeled condition…
2 Likes
I no longer head the group, we like to change leadership every few years. Some of the folks who help in setup are psychologists, physicists, audio engineers, performing artists, and statisticians. I am not qualified to speak on their behalf as far as methodology. When all is said and done, you know the scores you assigned and can listen again to specific units and as far as fatigue, that is part of life and listening in general. Compared to the final tally, I had two items reversed, numbers five and six, many consider this a training course on how to listen, and I try not to make claims about my own hearing abilities or lack there of.
I am new here and I have probably gone way off topic. Generic if you would like to talk about some of this privately I am more than willing to discuss it further, we do the best we can but I am always open to new viewpoints.
4 Likes
You’ve not offended anyone, and your input is very welcome. We do have other topics that may be more relevant, and can move some topics/items there. This happens fairly often, as conversations naturally drift. As long as the content is valuable, it’s a net positive. More serious problems arise when discussions move away from audio or become hostile.
7 Likes
