Oh man, I will ask the wife, we are only going to be there for the day. Granville Island and china town are priorities for her and the lil monste…the lil kiddo…I might convince her to let me stop by the Headphone Bar. What do think of doing a headphone meet-up, people could bring their gear and spend a couple hours listening, I would make the trip up for sure with with my gear. Could make it an annual thing lol be the Penny Arcade of headphone meet ups lol.
Damn. That’s sounds good. Probably should try to do this by location.
Yeah I’ve been wanting to do one for awhile…but didn’t really feel comfortable with the Reddit community lol. I’m in the Seattle area (Snoqualmie/Issaquah). I would be willing to drive up to Vancouver or relative distance to that from Seattle (for the Seattle location).
We’re like a 10 min drive from granville island. Headphonebar is about 20 mins from the island. We are doing the Grouse Grind pretty early in the morning tomorrow though so we won’t be around until the afternoon.
We are actually looking at doing multiple meet-ups in both Canada and the US in the future. Its something we’ve discussed at length for a couple of years and looks like something that could be realized this year!
Gotcha, have fun with nature’s stairmaster! If I can’t make it work this trip, it’s only a 3hr drive I could always make another trip up. I will definitely make it for a Meetup! Any good hole in the wall must eat restraunt recommendations?
I have used the points system in the past. For me, it helps work through a long list and sort on various features that are most important to my wants.
I agree that it would be nice to have an easy way to aggregate impressions about specific aspects of a headphone. Something I wouldn’t want to do is to have to comb through hundreds of pages of discussion to learn about specific aspects of a headphone’s performance, as I have to do on Head-Fi’s impression threads. That said, except maybe when talking about a specific known reviewer, I don’t think points are meaningful because everyone has different preferences and, frankly, levels of generosity. Yelp reviews are a great example of this. It’s not uncommon for me to see 2 or 3 star restaurant reviews only to read in the comments that the reviewer liked the ambiance, got good service, enjoyed the food and didn’t pay too much!
To me, the key to making a collection of impressions and reviews truly useful is to treat the archive as structured data rather than just a collection of free text. Concretely, that means that an impression/review can and should still include unstructured text, but should also include fine-grained impressions on very specific aspects of a headphone. I would suggest a category/sub-category taxonomy with categories and subcategories such as:
- Build Quality
- Other Accessories
- Instrument Separation
- Source Pairings
- Schiit Magni 3
- Feliks Elise
- Good for Genres
- Not Good For Genres
- Specific Song Impressions
- Yuja Wang Rachmaninov 3rd Piano Concerto
- Massive Attack Angel
- DT 1990
- … other specific models
- Frequency Response
- Impulse Response
Subcategories should be easy to expend (e.g. adding new amps). If reviewers have impressions about a category that don’t fit neatly in a give subcategory, they could just leave them at the category level. Each of these impressions would be subjective text and an optional numeric rating (since some people like myself may not feel comfortable putting numbers on stuff).
As an oversimplified (and slightly crappy) example, for my HD 600, I might provide the following:
- Build Quality - Not fancy but functional and durable
- Finish - Durable but not particularly attractive. The faux marbling isn’t as off-putting in person as it is in pictures, but I would have preferred a solid color scheme like on the HD650 or HD6xx.
- Headband - Combination of metal and plastic. Don’t bend it too far, as I’ve heard it can break.
- Yokes - I expect these will last a long time
- Connectors - The cable is removable. The connectors are a custom thing, but there’s plenty of aftermarket cables available with these connectors. They are a little tight to remove and insert, but that keeps them from falling out.
- Comfort - Overall very comfortable, I could almost forget I’m wearing them
- Headband - Very nice, no hot spots
- Clamp - A little tight, but loosens up over time
- Pads - Soft, breathable and very comfy
- Weight - Light
- Cable - There’s only one included cable and it’s too long for use on the go or even at my desk. I bought a cheap aftermarket cable on Amazon (they’re plentifully available from companies like NewFantasia).
- Weight - Heavy
- Tangling - Probably not too bad given its length
- Plug - Plug has a built-in adapter but it’s odd, the 1/4" slides completely over the 1/8". It took me a while to figure out that there was even a 1/8" in there!
- Other Accessories - Not well accessorized.
- Case - nope
- Sound - Overall pretty neutral though with elevated mids, a little grainy to my ear and not the best bass performance
- Clarity - Pretty good, although I often hear some sort of hash/grain that makes these sound a bit hazy compared to my DT 1990
- Timbre - Very natural
- Bass - A good bit of mid-bass weight and punch, not great extension. The bass distortion makes itself noticeable in that the clarity in the bass is not great (i.e. sounds a bit muddy).
- Midrange - The highlight of these headphones. Somewhat elevated, but not unnaturally so. Really shines on voices, guitar and piano.
- Treble - Pretty good, though it has a little more low treble than I’d like (can get slightly fatiguing) and less higher treble than I would like (lacks some sparkle and air).
- Instrument Separation - Not the best. Complex passages of orchestral music can sound like a wall of sound, with individual instruments difficult to pick out.
- Soundstage - I’m told it’s narrow, but I almost always use crossfeed so it’s of little interest to me.
- Source Pairings
- Schiit Magni 3 - Sounds fine.
- LG V20 - Sounds fine.
- Macbook Air - Sounds fine.
- Feliks Elise - (I’m making this up because I haven’t heard it). Greater bass weight though slightly muddier, less fatiguing treble
- Good for Genres
- Classical - The very natural timbre is great on stuff like Segovia’s guitar works, but I don’t love the congested sound on Beethoven’s symphonies.
- Folk - Excellent. Voices and guitar are front-and-center where they should be.
- Rock - Generally pretty good, though the faster more technical stuff isn’t so hot.
- Not Good for Genres
- EDM - Not so great. Tried listening to Massive Attack’s Angel with this, didn’t do much for me.
- DT 1990 - With their treble EQ’ed down, I vastly prefer the DT 1990 for it’s greater overall clarity, sparklier treble, and much more precise, better extended and punchier bass. The HD 600 pulls a head a little bit on vocal-focused music like folk rock, but for DT 1990 doesn’t fall that
- LCD2C - The LCD2C’s bass extension and clarity blow the HD 600 out of the water. The LCD2C also exhibits less of the grain and haze that I hear in the HD 600. However, the HD 600’s overall tonal balance sounds more natural and less dark.
The benefit to capturing this stuff in a database is that it opens up the possibility for all sorts of applications.
- Build Your Own Review - Select which aspects you care about and then see reviews include only those
- Focused Analysis - For example, see all impressions of the HD 600 with classical, or see all classical impressions for all headphones.
- Review the Reviewer - Look at all of a reviewer’s impressions of “bass” or “classical” to see how they tend to evaluate this and put their review in context.
Building this database wouldn’t be easy, would require a fair amount of work on getting the taxonomy right and probably require some iteration before it’s just right, but I think it would become a tremendously valuable resource. Especially with the ability to attach crowd-sourced measurements, I’m excited about what this format would bring.
Very nice post. There are some ways to handle this with a bit less structure, but the trade-offs are either
- Money or
- Time and expertise.
If you have money to throw at it, call in the experts:
If you have time and expertise, go open source:
I was a content data professional, now 4 or 5 years into recovery, having not kept up with the industry, done xml markup, built DTDs or Schema, or talked with professionals about structured data editorial systems in that time.
I’m going to stop now, because I don’t want to descend back into the depths of driving around, thinking about extensions to content management systems.
Andrew - some practical observations.
- unless you pay them, writers are like cats. Difficult to herd.
- even if you pay them, writers are like cats. Difficult to herd.
- it is possible to use two different databases - your forum here, and a second one that monitors it and builds the proper indices.
- Expect to spend a minimum of $30,000K and 500 man hours, or $80,000K and 200 man hours, depending on your solution. Even the open source will cost you your time and money to learn it.
Results, however, can be worth it. The links I posted above to solutions are generally considered top contenders. There are plenty of other players, but they often either try to herd cats (did I mention that was hard?) or use components of the above.
@pwjazz note my clever use of a numbered list in one spot and a bullet list in another, resulting in a style error.
I like the idea you’re presenting here, in terms of what it would enable.
I can also say, as someone that semi-regularly posts reviews of gear, some of them pretty extensive, that there is no way I’d fill all that out in the form necessary for it to work as structured data. And even if I did, absent a big guide, or additional context (both of which entail additional work on the part of the reviewer) you’ll still get subjective and/or incomplete responses.
For example, unless you have a range of values specified then things like “Cable: Weight” will be useless as what one person considers heavy won’t gel with another person’s opinion. And if you think I’m going to pull out a scale and measure that for every headphone I want to post a review or impressions of … well, let’s just say it’d result in me never actually getting around to making the post at all.
My impressions or reviews might include such things anyway, and I agree it’d be great to have them processable, but having seen how this sort of things tends to go when those having to do the work are faced with that much structure … my expectation would be that you’d get great data on very few reviews. Per @pennstac’s comment, it’s hard to get people to observe that much rigor even when you’re paying them.
And then the degree of time/effort I, personally, am willing to put into a post on impressions or a review is highly dependent on what item we’re talking about. I’m going to say and provide a lot more information about a $4,000 set of headphones than I am a $100 set.
Having a “Headphone Database” with such specific information in, that then links to the more common narrative reviews might service both goals. I think having a list of information that is good to include, with examples, such as that you provided, would also be useful. But I do believe making that a necessary part of posting impressions or a review will simply result in people not bothering and just posting their thoughts in the product threads anyway.
I’d add that, perhaps for gear made available via the Community Preview Program, you might be able to get people to fill out some of that … but I’m hesitant in thinking you’d get everyone to do it all.
I agree 100% that entering impressions and reviews should be easy and enjoyable, but I think that’s solvable with good UI design. Certainly none of the detailed categories should be required, like I think the following would be a legit review…
Build - utilitarian and solid
Comfort - lightweight and comfortable with an initially strong clamp that eases over time.
Sound - amazing mid-range, okay clarity, slightly bright with a bit of a haze and slightly muddy bass.
Source Pairing - Sounds fine out of my laptop and smoother and bassier from my OTL tube amp.
Genres - I like these with folk, rock and small ensemble classical
Comparisons - I like the DT1990 and LCD2C better because of their better bass performance and instrument separation.
One thing that this all reminds me of is a survey. In fact, doing periodic semi-structured headphone surveys could be a valuable way to collect impressions.
Lastly, as a reviewer myself I actually think having some structure helps remind me to look for certain things and actually makes it easier to collect my thoughts.
Also, I suspect we could use natural language processing to do feature extraction and sentiment analysis on free text. We’re dealing with an extremely specific domain with a constrained core vocabulary and fairly fixed scope of products.
I’m going to disagree here and say that your example is what I would call “impressions” and I would personally choose to leave those in the main official headphone thread. Going back to what I said earlier in this thread: “It doesn’t have to be super technical, but there should be some critical thought into understanding why the sound signature sounds “good” or “bad” to you. That really might be what distinguishes a “review” vs an “impression” to me. For me, impressions are more like bullet points of thoughts. You just say stuff like “this headphone has a really nice bass response that works well with electronic music”. Those types of comments are great for getting a feel of a headphone and whether or not to even consider it. But a full review helps me actually understand the details of the headphone and what it is doing such that it sounds a certain way.”
Let me break this apart why this falls into an “impresion” vs a “review” for me:
What are the materials that make it feel solid to you? Some people only think metal feels solid and that anything plastic feels cheap. Some people think good, scratch resistant plastic feels better made than metal that dents and is too rigid. Think old school plastic Samsung phones vs metal iPhones here. There are arguments for both sides.
Lightweight compared to what? Lighter than HD650 or lighter compared to LCD-3? Does the initially strong clamp make it painful or is it tight but snug and gives a nice seal (e.g. LCD2C)?
Why is the mid-range “amazing”? Is it because of the timbre? It is because it is on the warm side? Is it because vocals are very forward?
There is nothing wrong with your example and I am not trying to pick on you. That is exactly what I expect “impressions” to be. Those are the types of thoughts one could get from a headphone from listening to them for 10 minutes at meet. And those are the types of thoughts I would enjoy seeing in the official headphone threads. I think of it this way. If I am going into the official thread for a headphone I am curious about and haven’t heard myself, I want to see the general opinion on it and see what it is generally about. Those impressions you listed are perfect. That way I could quickly tell if it fits with what I am looking for. If I want a light, mid-focused headphone that would do well with acoustic music it sounds like from your description this is a headphone worth investigating more and doing some research on. And because I know you, I am assuming you were describing HD600, which all of your impressions sound dead on with how I hear it.
So from there, that is when I would start going to the separate “review” repository for HD600 and start digging in to learn more about the finer details and get a better sense of if this headphone really would be the best purchase for me. As an example, I posted the following Campfire Audio Cascade review in the official thread for now until a separate “review” repository is made. I only had the Cascade for about 6 days so maybe 15-20 hours of head time, and this is sort of the minimum of what I would personally write for a full review; my other reviews might be 2 times as long. But this is the type of detail I would expect to see from a full “review”:
I admit this is just my pesonal opinion of how I like to see separations between impressions and reviews, and I have no idea if this makes sense and works for others. If I am totally off base and people have better ideas than I am fully open to those too. But at least for me, that is sort of how I am used to approaching new gear and the methods I have used for learning about gear in this hobby.
Ha yes, my example was just to illustrate that if people want to leave low-effort impressions they should be able to. IMHO the difference between impressions and reviews is more one of degree than of kind. Quoting from a DT1990 review of mine:
As I’d hoped, the DT 1990 does very well on metal like Dream Theater’s “Metropolis, Pt. 1” and Rage Against the Machine’s “Bombtrack”, with a punchy and tight bass that does justice to fast double-bass drum and bass guitar work, a strong lower midrange that gives electric guitars a nice crunch and an elevated treble that brings out that guitar edginess that’s so fundamental to metal and which I miss with the HD 600 and LCD2C. The neutral midrange balance also allows vocalists La Brie and De La Rocha to sound clear, natural and authentic while occupying a balanced place in the mix. Lastly, the DT 1990’s dynamic and crisp presentation of percussion, plus a little sizzle, does justice to fine drummers like Mike Portnoy.
I could just as well have said “they’ve got very nice bass, neutral mid-range And sparkly treble.” This would be consistent with my review, just less well elaborated.
Yeah it seems are on the same page here. I’m fine with low effort impressions for the official thread. That’s the place for questions, answers, impressions, general comments, etc. And then the separate review repository would be reserved for lengthier, in-depth reviews like your DT1990 example.
What I would actually prefer is that impressions and reviews be their own thing and not just a thread. Head-Fi has those long impression threads which quickly fill with irrelevant side discussions and make it difficult for me to find actual impressions about the headphones themselves. I’d like something more like the Head-Fi reviews section, even for quick impressions, with the ability to easily filter through the impressions/reviews, look at them in aggregate, and so on. It would of course also be good to be able to discuss the impressions, but I think that would be better served by separate discussions in response to each posted impression rather than one big thread. And I think we could just tag the full reviews as “full review” so that if I want to just have those, I can filter for it.
Elaborating on the review vs impression thing, it strikes me that a review is really just impressions with context and supporting evidence. If an impression is “I like the bass on the LCD2C but wish it were stronger”, a review is more like “Hello, I’m Bob. I’m basshead who loves my TH-X00. I listen mostly to electronic music, especially jungle and drum and bass. I compared the LCD2C with my TH-X00 listening to X, Y and Z songs, driven from my Magni 3, volume matched at 78 dB, etc… I found that the LCD2C’s bass extension matches the TH-X00, but kick drums lack some of the slam of the TH-X00. I tried X, Y and Z equalizer settings and that improved things, but if you want slamming bass, stick with something like the TH-X00. Etc …”
When I think about how I consume such information, there are a few ways.
- Sometimes I just like to read well-written reviews because I enjoy sound and enjoy intelligent people writing about it thoughtfully.
- Sometimes I’m actually trying to make a purchasing decision and want to research some specific equipment. I’d like to look at it in aggregate to get a general feel for what people think about the bass performance, comfort, etc.
- Having found some general impressions, I might want to dig in some more to learn about some of the people who provided those impressions, what other equipment they own and their general listening preferences, and how they came to their specific conclusions (e.g. what did they compare to, what songs did they listen to, etc.)
Item 1. Is what you would call “full reviews”, 2. is what you would call “impressions” and 3. brings it full circle back to reviews. In an ideal world, this all just becomes different views into the same core information that allow me to pivot, zoom in and out and slice and dice to my heart’s content. If we could achieve that experience for the consumer of the information coupled with the ability for contributors to enjoy writing big reviews as well as be able to give quick impressions here and there as time permits, that would be my personal optimum.
That’s exactly what I am referring to when I say “review repository”. Earlier in this thread I specifically used Head-fi’s method as an example in that there is a standard, long running, continuous thread, what we here are calling “official threads”, and then a separate review area where full length reviews are written, and within those people can comment and respond on each individual review, but they are all grouped together in one collective place. That’s what I mean by review repository. So what I am trying to do now is define is what content would belong in the former vs the latter.
Generally for me, the way I approach new gear is I first start in the “official thread” if I know nothing about the gear and usually the very first posts will give a good explanation and sense of if I want to explore it more. Then once it seems interesting to me I will jump specifically to the review area and just read individual full length reviews. And then if I am pretty serious about that product but need some questions answered I will go back to the “official thread” and search to see if my question has already been asked, if not, ask it in there. And then once I actually buy and own that product I will usually continue to keep up in the “official thread” to answer other peoples’ questions or to just pay attention to how others are hearing that gear, how they are pairing with other sources, or mods they are making, etc.
So going back to my original point, those very short, bullet point types of “low effort” impressions that can be had within brief listening session would go in the “official thread”. But the full length reviews that involve critical thinking go into the separate “review repository”.
For me, Item 2 is usually covered in Item 1 if that is indeed a full review. A full review should be pretty comprehensive in talking about build, comfort, sound, source pairings, genres, etc. If you specifically want to figure out the bass response of a headphone the review repository would be perfect since you can read through each individual review and jump to the “bass response” section of it. You wouldn’t have to wade through the noise in the “official thread”. But the “official thread” should be for everything else including questions to specific people or the general audience. I also feel those short, quick impressions work well in there too as often times people give those impressions in response to a question about the gear. It follows a natural flow to give bullet point types of impressions/answers to people discussing specific things within the “official thread”.
Now, if you wanted to dive deeper into a specific full review and ask the specific reviewer more related questions to their review, I like how Head-fi’s system allows those comment chains to build within an individual review. So each individual review does have the ability to have its own related sub-thread, but all the individual reviews are collected together into 1 repository. That is ideally how I would envision the review repository working here.
Taking a step back for a moment and tabling my proposed solution, I generally agree with what you’re saying but believe that there are three things on which we can improve:
Increase the ratio of structured reviews vs discussion forum chatter. Continuing with Head-Fi as our touchstone, if you look at the Focal Clear, you see 3 reviews vs 3716 messages in the official thread. When looking for review-like information, it would be useful to have a greater diversity of reviewers represented, and I would personally be willing to sacrifice some verbosity in the review in exchange for more reviews. In other words, it would be nice if some of the longer/better impressions buried in the official thread were surfaced as reviews.
Assuming we solve 1, it would be nice to be able to get a longitudinal view of this large set of reviews without having to read each review in depth. The summarized pro/con sections in Head-Fi go some way towards this, but with a bit more structure I think this could become more powerful and useful.
The discussion in the official Head-Fi threads often veers into tangents that make it very time consuming to dig through them for information about the headphone that’s ostensibly the focus of the official thread. I think there are things we can and should do to avoid this happening here.
I agree on all your points. And here are some possible solutions:
I believe the reason for the lack of use of the “review” section of a given headphone in Head-fi is because it isn’t clear that there are separate section in Head-fi for reviews. I was browsing and subscribing to dozens of threads on many different products for literally years before I finally realized there were separate review sections. I even posted full reviews myself in the long, ongoing impressions threads because I never knew about the separate review sections. Those review sections just aren’t clearly linked to the impressions thread. I mentioned this at the top of this thread about why Head-fi’s system is good, but why it also has shortcomings. I propose that if we have a separate review repository section that it be directly linked and referenced somehow in the “official threads” such that they are basically tied to each other. And I also think it will take a bit of moderation to move full length reviews to the separate section if they accidentally get posted in the “official thread” area.
The structure issue is essentially what this thread discussion has been about. That is why people have been arguing for or against using a rating system or having templates. I am a bit undecided on this myself as I like having structure, but I also don’t want to force people to have to fill out templates, especially if the thought of doing so discourages potential reviewers from writing reviews. Maybe we can just use a “TL;DR” section at the top of all reviews, which really is the same idea of what “abstract” sections are in scientific journal articles. Something like a quick summary at the top of each review would probably help readers quickly decide if they want to invest a lot of time in diving into the full review or not.
I agree about the tangents, but I actually don’t think the tangents themselves are bad as long as they are on-topic. I think with long running, continuous threads there is just no way to avoid that type of conversation flow. And I actually think it is fun as an owner to just get to know other owners within a thread and talk to each other about how they use that product. But I do agree that Head-fi’s SEARCH FUNCTION is terrible, which is why it gets time consuming trying to find relevant information you are looking for. I think this forum’s built-in layout will actually alleviate many of those problems without even having to do anything extra. For one, try out the search function in this forum, it is pretty solid. It is much more fluid and quick in being able to jump to relevant posts. Second, I believe the way Discourse works is that once a thread accumulates a certain number of posts (can’t remember the exact number) it automatically builds in a summary tool that uses a sorting mechanism to show the most popular posts. A great example of this is the “Songs to test headphones with” thread. Click on that link. In the very first top post it is already showing you “Popular Links” so right away you can just click on links to new playlists that people liked. Next, you will see a button that says “Summarize This Topic”. It now filters out and hides less popular replies. And then if you scroll through the actual posts there are a couple things to look out for. First, on the lower left corner of a post you might see a grayed out “# Replies” label. If you click that it will expand to show who replied to that particular post and what they said. Similarly, if you check the upper right corner of a post and see the “reply arrow username” label it means that post was a direct reply to someone else, and if you expand that it will show you the conversation of what that post was replying to. Those features are really important IMO. Often times I try to do this in Head-fi by skipping the hundreds of pages at the beginning and jumping to the newest posts. But unless a post specifically quotes someone else inside their reply, you lose context of what they were replying to and you have to scroll backward in pages to figure that out. This system here is much better for browsing through long threads to find the most useful information.
Agreed that just making reviews more visible is a good first step.
Quick summary is a good start, though I suspect in practice I’ll find myself wanting more structure. An NLP approach for extracting structure from the free text might be the best of both worlds.
Yes, the semi-threaded, infinite-scrolling format of Discourse is a big improvement over Head-Fi’s forum in and of itself, as are the popular links and summarize features and the much better search engine.
Another cool thing about Discourse is that it supports plugins so as we learn more about how it works in practice, we could create a “headphones” plugin that provides some domain-specific functionality to take it the extra mile.
It’s late. I probably shouldn’t comment. But I’m going to.
If you want to create a structured data set cheaply, you must feed the authors templates, and the templates must enforce the structures. You want to do a review? Use the template. You want to have your impression kept for posterity, use the template.
Next, you need an editorial component. Machines are not going to substitute. You don’t want them to say “amazing midrange” by itself? The editor has to send it back to the writer for revision. This is something I know. I was Product Manager for an online editorial system designed to build structured content for scientific, technical, and medical clients including McGraw Hill, John Wiley & Sons, Reed Elsevier, Lippincott Williams & Wilkins, and any number of smaller players including professional societies. Please don’t use another headphone blog as your standard of comparison.
The above is why I suggested looking at systems that produce indices that try to impose some structure on less rigorous full text. You are both right in that a headphone blog has certain limited or common content that will make this easier.
But the real problem is the human one. You can get SOME people to follow your structure, but you will lose many authors out of frustration. And there are good reviewers that do subjective reviews. They may be a bit quirky, and probably won’t fit into the proper containers.
Speaking of the editorial component - realize that high-quality content contains elements of both peer review, and copy editing. You need a workflow and a process to move content from creator, through an approval process, and then into an editorial process that includes both content and copy editors.
I really don’t have a dog in this horse race. I don’t think I’ll write much more here, as I don’t have a heck of a lot more to say. If you want my opinion, please ask me, I’ll be reading this, or message me. My background may not be appropriate for what you’re hoping to achieve in this thread, but I’m willing to add my comments if asked.
Is there any progress on getting a review implementation up? I am going to try and put up a number of reviews throughout the summer and it would be great to get them put into a new home.