I’m not sure that it’s flawed, but you might think about alternatives. There is a logic to what you’ve been doing (cheap are both limited and forgiven for being cheap, while expensive products open more doors to quality but can stumble badly). Human rating scales are fuzzy and require all sorts of QC and validation.
No, opposite of a shill but we do want to know if a reviewer is a shill… 
I know jokes aside. That last chart interests me a bit. Would be nice to see how it looks if you combine populations into categories of: Personal Units, Review Samples, and Loaner Units and I guess meets/shows
I wanted to do that myself, but lacked knowledge of the details. So, I’m tossing it back to you for the categorical roll-up. That could reveal some bias or random luck. Again, these are pretty small samples for human rating data.
Send me free sample review units and you will get lower scores! 
This is what I wanted to do, yes. I can generate a bunch of after-the-fact explanations for this pattern. All of this is speculation:
Many, many ways to rationalize…
For personal products, you’ll see the ones I kept are the highest scoring ones (obviously), and then there’s some I bought along the way as I moved up and then theres the major regrets of buying flavor of the month hyped stuff.
But yes, agree with the bullet points!
hello Antdroid ,what IEM do you recommend for instrumental music ,i have few IEM’s which are sub 500 $ ,like Fiio FH7 ,FH5 ,thieaudio L5 ,Tin P1 ,audio fly AF-180 ,unfortunately non of them is able to represent the low end or the high end properly ,as compared with my Focal Clear and Audeze LCD X headphones ,please advise what IEM could pronounce the Hi-Hat symbols or kick drum with accurate decay .thanks
I’m not Antdroid, so forgive my presumptuousness, but my personal go-to IEM for instrumental music is the QKZ VK4 with either Sedna Xelastec or ePro Horn Shaped tips. For stuff like GoGo Penguin, Phronesis, EST, Tool and so on, I haven’t found better.
I need to EQ 8900 Hz down a bit, but it’s not strictly necessary to enjoy these. The bass is high quality, with good impact, actual texture and detail, and a little bit of mid-bass warmth but more of a sub-bass focus that makes them sound fairly technical. The lively treble makes percussion sound great and gives these a very open sound. What’s important is using fairly wide bore tips that seal well, and don’t insert too deep else the bass can start to get bloated.
If you have a favorite instrumental track, I’d be happy to give it a listen and give you my impressions on what I hear with the VK4.
I just bought the VK4 recently to see what the fuss was about. It’s a tad bright, but overall nice tonality and a stellar value at $20!
Thieaudio Legacy 4 is a nice pick for $200 USD, though there aren’t a lot of iems that do both strong bass AND extend treble high in this price range. The Moondrop Dusk sounds promising. I havent heard it myself, though it does lack upper treble for the airy signature just like most things under $500 (including L4).
I’ll have to think a little more.
Just for kicks and giggles, if you have measurements for them would you mind throwing up a comparison with the U12t and Dunu SA6 please?
Thank you android for sparing the time to answer me ,I have the L5 ,the problem with it is the upper treble is recessed ,so the hi hat cymbals doesn’t sound complete ,I feel some distortion in the fiio fh7 too in high treble too as well as the fh5 ,the audio fly af180 sounds clear but when the track become crowded it start to distort
Hello pwjazz,thank you for your prompt response and for your effort to help me finding the right IEM. The instrumental music that I do listen to are a little bit crowded with instrument for example a track called freedom for a musician called Havasi it has percussion with piano and hi hat ,so it covers the full spectrum,or something like (kaddish for superman) by a musician called Yom .the problem with the IEMs that I have that the sub bass doesn’t rumble or doesn’t gives you sense of deep bass like the headphones or the speakers,and the cymbals of the hi hat has very fast decay or some kind of distortion,so I can’t feel it completing the cymbals smoothly in a clean way especially when the music is crowded with many instruments ,please note that I have a couple of DAPs (fiio m11 pro & pioneer xd300) as well some desktop daca and amps ,like schiit jotunheim multibit , topping dx7 pro , smsl su8 , smsl sp200 ,thx 789 ,geshelli labs enog2 pro .as well I’m using 16 bit or 24 bit FLAC files as source .
I’m listening to Kaddish for Superman with the following (cool song btw):
On the Legacy 4, the beginning of the song sounds pretty harsh and gritty. The kick drum sounds clean but not very full. As things start to build, the clarinet sounds quite distinct and detailed, I’m able to hear Yom’s breath, but it’s a touch shrill. As far as soundstage, everything sounds a lot more closed in than I imagine is the intent with this song. Separation is actually quite good, but just lacks air. Hi hats and cymbals sound pretty good but not special in one way or another.
With the Starfield, the kick drum sounds noticeably punchier and fuller, though more mid-bass focused than sub-bass focused. Although the soundstage isn’t very large, I do get more of a sense of space and air. The clarinet sounds smoother yet still relatively detailed, I can still hear Yom’s breath, but the clarinet is less forward than with the Legacy 4. Hi hats and cymbals sound a little soft.
On the VK4, the intro sounds brighter than on the other 2. The gong has a higher timbre. What sounds like static noise with the other two sounds like wind on the VK4 (maybe that’s actually what’s intended in the intro?). Immediately there’s more of a sense of atmosphere. The kick drum doesn’t sound as full as on the other two but has more depth, but it sounds a bit further away. Yom’s clarinet sounds nice and detailed, I can hear the breath, and I notice valve noises that hadn’t jumped out at me previously. The clarinet is a bit forward of the kick drums. The bass guitar also sounds nice (for whatever reason I hadn’t paid attention to it on the previous two listens). Dynamics on the crescendo around 3:45 are strong enough that I actually have to turn down the volume a bit. Hi hats and cymbals are more sparkly and with less body than on the Starfield, and they definitely have a more drawn-out decay. The one demerit is that the timbre of everything sounds a touch less natural than on the Starfield or even the Legacy 4.
Switching to the Xelastec tips brings the VK4 a little closer to the Starfield. It loses some openness, the wind noises in the opening sound more harsh again (though still better than the Legacy 4), bass has more warmth and authority while still retaining its depth, and treble is a touch less smooth. I no longer hear the valve noises on the clarinet, and the decay of cymbal strikes is a little more blunted. Timbre of everything improved noticeably thanks to the increased body.
So, from what I can tell, this song is meant to be kind of moody and atmospheric, with pretty big dynamic swings, driven by a beat from the kick drum and electric bass, with the clarinet sort of noodling along on an improvised melody and hi hats and cymbals sort of punctuating things here and there.
For my preferences, the Legacy 4 was clearly the worst here. It lacks the air to properly convey that sense of atmosphere, the treble harshness kind of gets in the way of things, and the bass lacks the authority to properly carry the rhythm. The one redeeming quality is that it rendered the clarinet very nicely, so if I were listening to this just for the clarinet part I might pick the Legacy 4 as the winner.
The Starfield put up a much more respectable performance. Although it doesn’t excel in its rendering of the atmosphere, it carries the bass with authority, renders the clarinet nicely, isn’t harsh and has enough treble to get by. It’s also got the best timbre in this test.
The VK4 is the winner for me, and I prefer it a bit with the ePro tips. It’s the most open sounding of the bunch, the treble is smooth but relatively elevated, which really lets the hi hats and cymbals shine, and the clarinet is clear and detailed but pulled back a bit which helps create a sense of space, and the bass is present enough to keep the beat going. I do wish that I could combine the bass fullness of the Xelastec tips with the air and sparkle of the ePro tips… Okay, more tip rolling. How about the Spinfit CP100? I only have them available in large, which keeps the insertion shallower than I like for the VK4, still worth a shot though.
Aha! Okay, I still get a good sense of openness and treble detail, but the bass now has depth and authority without sounding overbearing. Everything sounds nice and smooth, with probably a touch less detail than the ePro or even the Xelastec tips, but I really like the overall tonality here. With the bass being a bit more present, the clarinet does get pushed back a bit more, but it works. Hi hats and cymbals aren’t quite as crisp, but they do have a nice long decay and they’re still edgier than on the Starfield. Oh wait, if I back off the EQ cut at 8900 Hz, I get back some of that treble crispness. All good!
So yeah, for this song anyway, Spinfit CP100 tips on the VK4 are my pick, hands down. Really any tips on the VK4 are enjoyable here, just different flavors.
Whatever IEM you go with, I think a big part of the reason that the VK4 works well with instrumental stuff like this is that it’s a bit laid back between 1-3KHz, which in general helps open up space in the presentation. I find the same effect with Hifiman headphones. The price that you pay is that vocals and lead instruments like the clarinet here get pushed a little further back in the soundstage, which doesn’t always complement the recording (e.g. if I’m listening to something like Adele I want her in front of the band).
Ah the pains of addiction. For fun, I tried this song with one more IEM, the humble Sony MH755 with Xelastec tips. It’s got a pretty forward midrange, and sure enough it bring the clarinet further forward in the mix. It’s also got some massive bass, which is kind of fun on this song, and it’s got plenty of treble too. It does lack some air unfortunately, but it still sounds more open than the Legacy 4 and I’m enjoying it with this song, even though the presentation is very different from the VK4.
@JAKOB Not to confuse things, but for completeness I tried the ePro tips on the Starfield and that might be the best combination yet. It’s got more bass warmth than on the VK4 and almost as much depth, less treble but still plentiful, good decay on the cymbals, good openness (though a bit less than the VK4) and it keeps the Starfield’s good timbre and keeps the clarinet a little more forward, closer to what I’m guessing is the intent of a song whose star is the clarinetist. I do still EQ up 11.5 KHz to improve the sense of openness and cymbal realism.
It seems that a good bit of the character from the ePro tips carries over from the VK4 to the Starfield. So, before you do anything else, I’d say get yourself some ePro Horn shaped tips and see how those do for you with your existing IEMs.
Okay, off to bed with me now.
When did Superman die, and stay dead? Kal-el doesn’t even sound Jewish. 
Nice set of impressions, It’s a shame none of those Xelastic or ePro Horn tips don’t have wider bore options.
This gives rise to the conversations possible in Smallville, with young Clark and his Mother. “Oy, you’re always bringing home shiksas. Yes, Lana is cute, but the Jewish girls you don’t like? And no more playing sports or running after crooks on shabbos. Take this casserole over to Mrs. Goldfarb, then you have to practice the violin.”
@pwjazz ,WOW ,i really appreciate your effort and your enthusiasm ,for the love of music .i would like to mention that i do use dekoni foam tips on all my IEM’s because the silicon tips doesn’t give me a good seal ,maybe its tile to try different tips ! ,could i ask you what kind of source you’re using ? and did you managed to find or to listen to that album called (Drum & Piano project (Endi & Havasi ) Freedom ) track the Dual or Freedom or the storm or Drum and piano .if possible .what did i notice that all my IEM’s are doing the bass well on that album ,the difference between the Fiio’s and the L5 that the Fiio has more air all over the range .the L5 sounded very closed .the problem with all these IEM’s are when it comes to the hi-hat when the track become crowded with Drum ,Piano & hi Hat .honestly speaking ,i’m a computer guy ,not a sound guy ,so i don’t know how to explain it in details .
