I've made some progress. Firstly, this conversation is not new:
Secondly, the headphone output is 63ohm so according to this, the headphones need to be 200-600 ohms to sound good. All of my headphones are low impedance.
If I send the sound through my audio interface (scarlett solo) and plug the headphones into that, the sound is much better, despite the extra D/A A/D conversion. With my POD 2.0 the opposite was true (sound was good with headphones in the POD, less good via the audio interface). The scarlett solo headphone impedance is 10ohm, which is a better match for my 32ohm headphones.