Hello All!
This is the official thread for discussion of everything talked about in the ‘Myths about measurements’ video, and anything related to this topic.
If you’ve not seen it already you can watch the video here:
3 Likes
How about some biometric measurements? How big is @Resolve’s head? And the rest of you? What does that mean for measured clamping force? What change in clamping force does ± 1cm make? Is this linear?
Are there any measurements for headphone seal? How could you measure? Perhaps several sizes of tubing to simulate glasses, and a pressure gauge.
Yes I’m being a bit silly again but the point is that so many gross physical considerations mean much more than phantom measurements.
1 Like
The biggest of course… Actually I think Cam’s may be bigger.
And yeah, these are all variables that you don’t typically see with single line FR results. Clamp force in particular is a significant factor that contributes to some of the non-HRTF related HpTF variation we see across people.
Cameron noted this in the comments, but even for the HpTF variation we’re plotting now, those still use anthropometric pinnae, which means that to the extent there is variation, that’s not intentional. These rigs are all trying to be ‘standard’. Real human heads/ears are going to vary more substantially. So while we can get a sense of how a headphone may vary, we don’t yet get the full range. It’s mainly to be used as a signal that “headphones do vary across ears and heads” and also that some headphones may behave more consistently than others.
I would think that clamp force could profoundly affect HRTF, as it could compress the pads against the ear. And with many headphones, the way to adjust clamp force is to bend the headphone overhead connector (whatever it’s called) until it feels good.
Given that - and manufacturing variables, packaging, etc. it’s a wonder that we don’t see more variation.
Well no the HRTF for a person would stay the same, since HRTF is the response of a head and ears in a given condition (what the brain expects sound to be at the eardrum given the person’s anatomy).
But certainly headphone behaviour is impacted by these factors that are different from person to person, and we call this HpTF variation. And there’s all kinds of stuff that could impact the response at the eardrum, including clamp pressure and pad deformation, or situations where the pinna gets slightly impacted or deformed - though maybe we need a more specific term for that.
1 Like
And this is where the headphone cable debate becomes hopelessly confounded, such as:
Image of this beast of a neck strength builder and pad flattener:
2 Likes
Great video Cameron. Learned a few things about DAC and AMP measurements. It’s a long but it’s clear and concise, as long as it needed to be.
I wonder if the SINAD measurement being a simplistic default that some fall back to is because most solid state DACs and amps are simply past the point of being audibly transparent? When none of the measurements point towards any audible difference because we can’t hear -100db, then none them really matters other than having a d*ck measurement contest with SINAD?
In terms of headphone measurements, what you guys are doing with the new graphs and presentation is great. Perhaps on your measurement graphs you could add links to pages that explain what all these words mean:
Oh and maybe add measurement graphs for all the headphones you’ve tested, or at least fix all the broken pages like this one: Moondrop's BEST Over-ear Headphone - Moondrop Cosmo Review – Headphones.com (there’s nothing inside? or am I blind?)
From a personal stand point I wish something else could be tested, but I also understand why it’ll never be done. What I’d like to see is an additional graph that shows the distortions of headphones after it has been EQed to match Harman (or whatever listening curve) perfectly on the test rig (ignoring the wave cancellation dips that cannot be EQed). I know, not that useful for most people perhaps, it’s just something of great personal interest.
Like Cameron said, it’ll take a lot of effort to push the conversations forward so more things which are relevant to how we enjoy music and listen with headphones could become measurable. It is immensely frustrating to me that certain audible effects like sound stage, imaging, “punchiness” “tactility” continue to be described, heard, but not quantified (other than by rtings, and the audiophile community only likes to sh*t on them without providing viable or better alternatives of measurements).
Appreciate you guys making great videos and taking your time responding on the forum.
1 Like
Thanks, I’ll see if @listener can fix that.
1 Like
From what I’ve seen, it tends to result in one of two misconceptions:
- People thinking SINAD is an overall goodness score and making purchase decisions based around that, without bothering to look at the individual metrics to see if it actually will be suitable for them with regards to noise or distortion. The higher ranking = better confusion is common, and despite some on ASR claiming not to use it that way… they do commonly use it that way.
- People not realizing that SINAD is actually confounding the issue, and think it’s still a valid metric as a way of saying “at least beyond this threshold you won’t hear any additional products in your music”, which is also not guaranteed. Cameron’s three sound samples does a good job of demonstrating that a SINAD score creates false equivalence, but as someone else noted, it could’ve been done at a much better score to be a bit more realistic.
There may be a disqualifying aspect to SINAD if the results are quite bad, but even then… you may want to disqualify products that score much better if you have more sensitive IEMs for example. And at worst, SINAD is effectively a different kind of snake oil, where it causes people to make purchase decisions thinking they’re getting better performance when in fact they aren’t.
1 Like
I understand what you’re saying, however I’m not as well versed and researched as you guys are on amps and DACs so I cannot think of an example of this IRL.
Would you mind pointing out some products where the SINAD is really good (like… 110 or higher?) but the product still has be audible harmonics or whatever else? Would make for an interesting case study on the topic about measurement myths.
I’m only asking because I am one of those “at least beyond this threshold you won’t hear any additional products in your music” people.
I had to edit my post on this after it was (rightly) pointed out the sound samples were a bit of an extreme example. But it’s done so to illustrate the point. Realistically there are a lot of products that score well - below what many would consider audible - that would still yield audible noise floor with certain sensitive IEMs. In a way it’s less about the SINAD score of the device, and more about which product you’re using it with. But in that case you’d want to care a LOT more about noise specifically as a disqualifier than the SINAD score.
Now some of us may want to say “IEMs don’t count”, but I’m not ready to omit an entire segment of the community
Are there any examples you could link to? I’m asking not to shame any particular product, but I want to learn to read measurements other than SINAD which could indicate a potential issue even when when the product has good SINAD.
Like I said I don’t know the breadth of product lineups out there as well as you guys do, that’s the only reason why I’m asking. From my very basic research, it’s difficult for me to not be someone who thinks “at least beyond this threshold you won’t hear any additional products in your music”.
Thanks again Andrew
I actually think @GoldenSound may be the better person for this since he keeps up with source gear more than I do. But for sensitive playback devices, the CFA Andromeda is notoriously sensitive, as are several others. Yes, that’s bad design, but not just a one off.
Funny I just watched this video.
Really well done I must say.
I already knew most of the stuff, but there was stuff I did learn and it was concise and what I think the executive summary was, that we need to discuss more, and not yell at each other.
As in, it’s inaudible you don’t know what you are talking about. Or in reverse , this makes a difference where there is not enough evidence in it, or the controls of a study are flawed and not peer reviewed.
I myself use measurements usually for the inverse. To see if something is fundamentally wrong with an audio device or headphones.
Like if an amp has very high odd order harmonic distortion. Or has high noise from the power supply. Just as an example.
Or if a pair of headphones have exceedingly high distortion, or a freq response that is super wonky or has very high peaks or worse dip, which can’t be EQed.
And for headphones the more I try, some FR measurements will tell me how much I might like them from other headphones I like with similar curves.
I also EQ all my headphones so I also like to see headphones that have a FR that might be easier to add EQ to.
Headphones are wonky though as has been discussed over and over and for anything over around 2khz I need to do sweeps to see how I’m hearing something.
Just as an example a particular headphone was measured in a bunch of rigs showing a peak at around 5900hz. When I did sweeps I heard it at around 6700hz. Go figure.
But it gave me the knowledge on how to apply EQ more accurately.
The exciting thing is the more time goes by, the more we do learn. And of course money and resources dictate a lot of how things will progress.
Which brings me to the point he made about the dip at around 2khz for improved soundstage.
I’m not sure, but I think that the BBC actually did some studies around that. After all it is called the BBC dip. But I’m not sure if I actually read it somewhere that, that is the reason many British speaker manufacturers would add that in their speaker response to make the sound more ‘ natural’ from a soundstage perception. So I could be wrong.
Good vid
Oh and on another note. Headphone amps are weird. Lol
Some measure super well and sound flat and dull and others measure well and sound great. I have no clue what’s going on there.
I’m a benchmark fan, but for whatever reason I prefer the sound of the Ifi Zen Can to the Benchmark DAC3 headphone amp. I have no idea why. But the headphone amp on the benchmark sounds more dark to me and not as open as the Ifi. I don’t get it. I wish I did as to make more informed purchases in the future without having to try a million things. Well that’s why like I said I use measurements mainly to see if something is seriously wrong with a piece of audio gear. I could care less if SINAD on one device is 2-5db or whatever better.
Parodn for the long rant
1 Like
I salute all the folks who “run” Headphones.com and affiliated resources. You have an insurmountable task. How do you describe the quality of a listening experience?
It’s impossible. Its impossible for men to experience child birth. That’s kind of like audio, you have to hear these things for yourself. I own three IEM’s which graph similarly. A KZ SAGA, 7z Zero, and an ARTTI T10. Of course not identical on the graphs on squig.link, but similar enough.
But after months of listening, and tip swapping, and just far too many hours spent comparing. No contest, there are no words to describe that the ARTTI T10 is in a completely different league, so much more life like.
I bought the Zero 2, cos I read the super low distortion numbers, in AmirM’s measurements of the Zero 2 on AudioScienceReview.com.
There are no measurements to describe, how much better the ARTTI T10 is. In its ability to render a lifelike image, that is laid out in front of the listener, especially when listening to very well mixed audio/music. Nothing that we have so far. If I was a betting man I would expect the comparison of impulse response, to tell me the whole story.
I sympathise with any reviewer, only way to discover what sounds better, is to hear it for yourself.
So for DACs I think measurements have a lot of value, in my experience, up to a point, above which any further perception of clarity from a better measuring DAC, should not be audible. but for headphones/IEMs, the FR does not tell me which IEM sounds better than any other, where their FR is similar.
Let me put this in context. I have been an audio engineer, for almost 3 decades, but nothing could have prepared me for how good the T10 sounds. Nothing. No measurement can describe the improvement, over every other device I have ever heard, and that includes speakers.
This is the dilemma that faces audio commentators, and reviewers. Even I who write this, you who read it , will not believe me, unless you hear the Zero 2 side by side with the ARTTI T10, and also do your own tip rolling, and take time to get the best fit, then spend at least two weeks going back and forth between the Zero 2 and the T10. I say this cos, in my experience, it appears that the ear itself needs training, to appreciate what “better” sounds like.
I salute all the efforts on frequency graphs, to “normalise” them, but these revised graphs cannot explain resolution, clarity, and why a T10 sounds so much better than the Zero 2. Nothing in the world can.
With IEMs specifically, honestly there’s a lot of products that with more sensitive IEMs can have audible noise.
Firstly, some IEMs just have ridiculously high sensitivity. And so the same level of signal or noise from a particular amp may be 20dB higher or more on some IEMs vs some headphones.
There are also slight challenges with how SINAD is tested, since DACs aren’t always normalized to a single volume, so two DACs with the same SINAD but different output levels have different absolute noise levels.
And often when testing amps they are not in their lowest gain setting as they can’t get to 4V, so the SINAD score there doesn’t necessarily represent what noise level you’ll get when actually using them with IEMs in the lowest gain.
A solution for amps is to do the 50mV test, which is fairly regularly tested by myself, ASR and a couple other places. This way the amp is fed a fixed line level, and you adjust it to 50mV output and compare the resulting SNR to other amps, rather than doing 4V output.
This gives you a better representation of where your noise floor will be when running really sensitive stuff.
As to specific examples, to pick something at somewhat random, let’s take the Violectric V550. Assuming your DAC completely exceeds the amp’s own noise performance and is not limiting things, it has a SINAD @ 4V of about 110dB:
But at 50mV output it has an SNR of 78dB.
If we take the Campfire Andromeda @Resolve mentioned. This has a sensitivity of 112dB/mW and an impedance of 12 Ohm.
The IEM if you were listening at say 90dB, which is pretty normal, and leaves us 16dB room for crest factor/peak level vs RMS level (or indeed perceived level/LUFS since most places recommend about -18dB LUFS as a target) we would therefore have an SPL noise floor of 28dB.
This is already easily audible if either nothing is playing or the track has a very quiet part.
The difference between absolute maximum level and general loudness is a challenge here as it’s track dependent. But try listening to a normal piece of music, then play a 0dBfs sine without changing volume, you’ll likely blow your ears out.
And this is with a ‘good’ amp and a ‘normal’ track. There are plenty of amps out there with 10-30dB lower SNR which could get your noise floor up to 30-50dB below the general level of music. And if you decide to listen to higher dynamic range tracks like something from Hans Zimmer (The first part of ‘Mountains’ is -35dB LUFS) then suddenly you could find your noise floor is now getting close to being the same level as parts of the track…
Easiest way though to see how audible it can be is to simulate it using a tool like https://distortaudio.org/
Grab a track and put it into the tool, disable the harmonic distortion modifications but enable noise and set a level of -78dB to simulate listening to something like the Andromeda on that V550 amp.
I’ve done this on a part of Avicii’s ‘Lonely Together’ as an example of a more typical loudness pop track and uploaded it here: Lonely Together -78dB noise.wav - Google Drive
(Can’t share the full song for obvious reasons, but you can use the tool to try this on anything)
The first 20 seconds is the song, and the last 10 seconds is just silence. But there’s a -78dB noise floor applied throughout. So you can both hear what the noise floor sounds like during the track, and how audible it is once things stop playing or if there were to be a much quieter part of a song (or quieter song overall).
The noise floor is obviously audible. Even more so on IEMs due to the additional isolation.
2 Likes
I guess a quick way to summarise this would be that for really sensitive headphones and IEMs, full scale SINAD often won’t tell you much since what you really care about is the noise floor, not distortion.
SNR at 50mV (or THD+N since on the majority of products you are limited entirely by noise at these levels anyway) is a more ideal figure to look at.
And it can often be surprisingly audible particularly with higher dynamic range or quieter tracks.
When you’re looking at running much harder to drive headphones like a Susvara or something, then the noise floor is usually not a concern at all and instead you want to be looking at how the amp distorts when outputting high levels into a low impedance load, and/or how the distortion vs level curve is behaving generally into lower impedance loads.
2 Likes
What a great reply @GoldenSound , that’s exactly what I was looking for, thanks very much. I can just about hear that -78db noise with my open back in a somewhat noisy background. It’s definitely loud enough to be a problem on closed back and IEMs.
And how does the noise from a DAC interact and changes things on top of the amp’s own noise?
Also when people talk about sound stage or punch from different amps, in your opinion, is that just a lack of current measurements available to describe this or a sighted test issue where current measurements should account for any such difference?
1 Like
Generally a DAC’s noise floor is likely to be lower than an amp’s noise floor. Both due to DACs typically being higher dynamic range/lower noise inherently (easier to do with small signal devices that don’t need to be designed to accommodate high current outputs or stability into low impedance loads), but also because with the DAC, you’re typically utilising its full dynamic range. Whereas with an amp, as shown by the difference in SNR at 50mV vs full scale, you’re usually heavily attenuating the signal, cutting dynamic range and so the amp is almost always going to be the limiting factor unless you’re using a tube DAC or something.
Sighted testing is ALWAYS a big factor that cannot be ignored, and no doubt many impressions are simply down to that.
But in my experience, in the instances where amps have shown differences in stuff like that and I could ABX between them successfully, there was always something measurable to explain it.
Sometimes pretty obvious differences in things like THD vs Freq, most commonly amps with rising distortion into higher frequencies leaning toward a softer sound. But also sometimes THD vs level. Amps which rise in distortion as they output more current can sound softer on demanding headphones sometimes like the Susvara, even if their max current rating figure technically means they can get them to high levels without outright clipping. Although this is much less of a thing with dynamic driver headphones.
2 Likes
I really liked this video, balance is needed in these times.
Mike
1 Like