This is an over simplification. What this references is damping factor between an amplifier and transducer.
In OTL tube amps, the output impedance is usually high. In amplifiers using output transformers or solid state power sections, output impedance can be controlled. That is why most OTL amps are considered bad matches for low impedance headphones, and why non-OTL tube amps CAN be ok, mostly.
If the amplifier cannot control the transducer, you get flabby bass and lose resolution in other frequency ranges. Some people do actually enjoy this in some amp / headphone pairings, as the distortion can sometimes be perceived as pleasant. Usually not.
It’s about that simple. Only not.
Let’s compare two headphones: the HiFiMan HE6 and HD650. The HE6 is a 50 ohm headphone, where the HD650 is 300 ohms. It would be easy to compare 50 vs 300 and think that the HD 650 is a harder headphone to drive. That is not the case.
Some headphones require an unusually high amount of power just to adequately move the transducer in the first place. The HiFiMan HE6 is a classic example. This difficulty is directly related to a headphones’ sensitivity rating, or in other words, how efficient the transducer is in converting power to sound.
Leaving impedance out of it for a moment, if the amplifier cannot supply the power needed to drive the transducer to satisfactory volumes - and often having extra power / or headroom is desired as well - then the sound isn’t going to be stellar either.
Sensitivity is rated in two ways: dB/mW SPL, or dB/V SPL.
So, the HFM HE6 has a sensitivity of 83.5dB/mW. The HD650 is 100.5dB/mW. The more power that is applied, the louder the HD650 will get compared to the HE6. The HD650 is more efficient.
Because the sensitivity of the HE6 is so low it requires a much larger power source to get suitable volumes, AND because it’s impedance is so low it also requires an amplifier with much lower output impedance to control the driver accurately, this makes the HE6 one of the most challenging headphones to drive properly.
That’s why people use speaker amps to drive the HE6.
Combine sensitivity and headphone impedance to get a clear picture of how hard a headphone is to drive.
I hope that helps.
Enjoy.