Not reaching the 2V “standard” output from a phone, or DAC/amp output (like the DragonFly Black for example, which caps at 1.2V), means that you’ll be able to utilize less of the available power in the amplifier you’re feeding.
The same would be true if your “line out” didn’t reach 2V (some don’t).
Next, headphone outputs are intended to drive relatively low impedances (typically between 16 and 600 Ω, where as line-outputs are usually designed to expect much higher impedances (typically 10 kΩ or more). This is less of a problem than if it were the other way around, but isn’t ideal.
Most importantly, typically, is that “headphone outputs” are driven by an amplifier. ALL amplifiers generate noise and distortion. This is usually level-dependent. So as the level goes up, which it will to try and hit that 2V goal, so do the distortion figures as well as the noise floor. Which means when you feed that into another amplifier, it amplifies that incoming noise and distortion AND adds its own.
Generally line-outputs are buffers, rather than amplifiers, and have MUCH lower noise and distortion performance than an amplifier operating at half, or more, of its rated output voltage.
Again, using the DragonFly Black as an example. To get as close as possible to the 2V spec, it is running flat out, at the limit of its voltage rails, which is where it performs the most poorly (no amp running against its rails can be expected to be as clean as when it’s just “ticking over”). And in doing that, it’s hitting just 1.2V.
This means that a) you’re getting the highest noise and distortion from the DragonFly Black that it’ll ever generate and then b) you’re feeding it into another amplifier, still only giving it half what it really wants to see so that c) it can amplify that more-distorted-than-necessary-and-noisier-than-it-should-be signal to appropriate levels while d) still being unable to reach its rated output (which is typically specified for a nominal 2V input signal).
It won’t break anything doing this … but it will not sound as good as it could and compared to most other differences in an audio chain (e.g. say comparing an SDAC to A Modi 3), these issues can be relatively gross and easily audible - often unmissable.
TL;DR; Don’t feed amplifiers outputs into other amplifiers.