No like, ‘HRTF’ is not a proxy for just “how I hear things”, which I think maybe some people may use the term. So it’s a response at the ear drum based on a given condition - we use the diffuse field sound field for headphones because that’s the one that makes the most sense given headphones are worn on the head, and not like speakers at a distance. But that’s also a calculation from lots of different incidences.
So like, if you change the angle/position of a speaker, and you measure it at the ear drum, the FR result will change quite substantially as you change the position of the speaker. But at the same time, your head and ears are impacting the sound every time, and this could also be considered the “HRTF”, just not in a diffuse field, which would include all/no direction.