This is a crazy thought I have had since starting my headphone journey: crowdsource based headphone comparisons.
This thread is for brainstorming and perhaps collaborating on such a project.
3 Likes
@Tom_Ato here it is.
Youâre fast! 
Indeed. Will try and organize and write up my thoughts in the next day or two.
1 Like
Headphone advisor
What problem does it solve?
Picking a headphone is a personal choice. Even when you know exactly what you want, filtering through everything can be exceedingly challenging.
The goal is to make this process as simple as possible for both beginners and advanced users.
How can it do that?
There are a couple of methodologies ranging from simple to complex:
Objective filters
These are simple things like price, closed or open, IEM or over ear, brand. Etc. I know of very few sites that act as a cross brand database to find headphones even with objective filters. The primary goal here is to have a large database of manufacturer supplied info for each headphone.
Now the complex bit:
Crowd Source data
Many reviewers make tier lists of the headphones that they have heard. This site would allow everyone to rank the headphones they have heard against other headphones across categories. For sound stage: put them in order of best to worst. Same for detail. Etc.
The idea being that there could be hundreds if not thousands of rankings for each headphone. And that specific users preferences would also be captured in that data.
What can be done with this data?
On a simpler level, you can make generalizations about headphones on a statistical level. For example, X percentage of people think Arya has the widest sound stage. Or, strengths and weaknesses among one or more headphones.
How can it be personalized?
Again, on the simpler level, a user could filter by objective data, and then among those find headphones that most users say are strong in a selected area. Maybe even multiple selected areas.
The users own rankings and favorites could eventually be used to find similar headphones among pools of people who generally agree. This will hopefully make it useful to those that have preferences that donât follow the masses. Arya drives you up a wall? Cool, 27 other users agree with you and here is what they like.
Long term ideas
Pairings. At some point, pairing info may be collectible, but it may be that there simply isnât enough data to talk about X headphone with Y amp to create meaningful predictions. But such things could be collected and shown individually.
Flagging professional reviewers so they can be filtered on and weighted. Relying on experts or weighting their rankings higher may be reasonable. This could include databases of links to peoples reviews (professional or not), youtubes, etc. This also gives incentive for reviewers to be involved with the site for exposure reasons.
Integrating âwhere to buy themâ through selling partners may also be a good thing. (particularly if such a thing could fund hosting). Again, retailers involvement for exposure.
And, one more crazy thought: Loaner tour management. I have some specific thoughts on this, but it could easily become its own product or module for effort. I think this would be generally valuable to the community.
Challenges
For this to be effective, there will need to be a lot of data collected both objectively and subjectively. Ideally, only those people that have actually heard headphones should rate them. âProvingâ this may be difficult (trust system may be required), but with a large volume of good ratings, the bad should be naturally drowned out.
Making sure people understand the different categories they are rating on could be tough. My current thought is to prepare samples of relatively extreme versions of each sound effect that are artificially enhanced to show what a category is expecting (and understood even on crappy headphones). This could even be built as a âquizâ to make sure users understand the intended meaning of the ratings.
Obviously, building the thing. Doing the above requires web/software development and machine learning. Collecting the data is probably the easiest part of the site. Using that data well is potentially challenging.
With this pool of data, there will be a lot of things that could be done. It will be necessary to keep narrowly focused on the most effective things to do.
Funding necessary hosting. It needs to live somewhere. And there isnât an immediate revenue opportunity. Hosting could be a problem.
Conclusion
These are my initial thoughts. Itâs been rattling around my head for a while.
I am open to any/all ideas. The first goal would be to pick a path of least resistance and build an MVP getting a lot of boiler plate out of the way. (Stuff like DB setup for basic info and letting people drag and drop rate, etc. This is not hard per se, just work).
Happy to have a discussion! And potentially organize help.
5 Likes
Is there an easy way to take a bunch of frequency response graphs and objectively turn them into some sound signature categories and sub-categories? For instance, taking the Harman curve as a baseline, categorize the bass, mids and treble as â+â , â-â or â=â against that.
And then you could also have crowd-sourced sound signatures too, similar to Crinacles categories (âBright neutralâ, âWarm neutralâ, âBass-rolled neutralâ, âNeutral with bass boostâ etc).
3 Likes
This has been on my mind. Assuming you could get the FR graph data, itâs entirely possibly to do something with it.
the problem I see there isnât actually applying the technology. Itâs enough people actually understanding what they want in an FR graph and selecting it.
From a UX thing, it would be easier for a user to say âI like the bass on this headphoneâ and have the software figure out how to use FR graphs as a filter.
But, yes, itâs on my mind. One of many things that didnât make it into the things above.
1 Like
This is an interesting idea.
Iâm sure Iâm not telling you anything you donât already know, but in my opinion, the best way of figuring all this out is to limit your initial scope, go ahead and build a POC (proof of concept), let people play with it and give you comments/complaints.
On the data projects at work, I lose patience when people start with a blank sheet of paper and spend weeks (sometimes months) arguing about the minutia, so Iâll often just mock something up in a spreadsheet and say âwhatâs wrong with thisâ, which helps focus them. It doesnât bother me if I get a lot of abuse, as long as the abuse is descriptive enough that I can create some requirements out of it, e.g. âthis is a piece of crap because xyzâ.
2 Likes
On it. See second to last paragraph of challenges. That is essentially what I meant.
Honestly, for MVP, I wouldnât even include machine learning. Just basic DB statistics over the data that is gathered. Simple ranked lists of the results being gathered. Etc. Probably far more focused on a smooth UX so people want to use it.
Basically turning MVP into a login/data gathering job.
I just want to know where I am going so I donât gather the wrong data or store it in a way that makes it hard to use. Thatâs it.
1 Like
A very interesting idea, but a monumental task to assemble and organize.
Perhaps you could make a deal with Crinacle to license his extensive database of rankings of IEMs and headphones, and add fields to it that incorporate opinions from the forum.
4 Likes
Currently I am not really thinking about monetization much. Licensing is probably out. But, I agree, crinacleâs database would be golden for any FR response feature.
I would consider such features to be âpost initial successâ.
Right now, I would prefer to think about it primarily as a community tool than a business. But, ultimately it will need to have some income to be sustainable over the long term. (Or a generous patron following)
I am open to all the ways.
3 Likes
I donât think so personally. Graphs only tell you so much and the Harman curve is based on 200+ individuals. Every person has other ears and preferences.
I would add the option to describe your favorite sound by selecting attributes.
1 Like
Since Crinacle doesnât charge for viewing his IEM and HP rankings, perhaps he might consider licensing them to the forum for free if you included a prominent link to his Patreon page and didnât charge for viewing the forumâs version.
3 Likes
Yes, I agree with you. Iâm suggesting using the Harman curve as one data point, to be used as a baseline that everyone can relate to, even if they donât agree with it. This would be combined with other data to provide a more complete picture.
3 Likes
That I can do. Also, I am technically not interested in displaying graphs. Just using the data. I would be happy to link to crinâs graph for that.
Crinacle does a lot more than graphs. Here is a tiny example of his headphone rankings database:
A key advantage is that you can sort it on any field.
Same thing for IEM rankings.
Much of this sounds very familiar to the website we are using right now? 
3 Likes
Good thought. This might be relevant: AutoEq/results/RANKING.md at master ¡ jaakkopasanen/AutoEq ¡ GitHub.
Yes he does. But the things he does beyond graphs are highly personal. If I had listened to his first impressions of the Zen, I might not have bought them. And not buying them would be a mistake. (They came out after I bought them anyway, just an example)
I use crinâs rankings all the time for research. But I definitely diverge from him too much to trust them beyond getting an idea of a particular products quality.
I compared the ath-r70x early in my journey. I chose it mostly based on crinâs rating. Compared to the sundara, I would put the sundara above the r70x in every way. And thatâs my opinion, and I am sure some subset of people would say the reverse.
I want to provide a tool that attempts to avoid the pitfalls of single or even subset of reviewers. Even great ones.
I believe we need to establish attributes that define the general sound signature of any headphone in the database.
Also, we could have attributes that describe how the devices perform in various frequencies and things like stage, separation etc.
It would be further nice to know what music genres the devices are good for etc.
Not sure if a rating is necessary. Also ratings are very very personal abs thus a bit abstract, especially considering different price points.
1 Like