You say with your headphones it works fine.

Check the volume with a new empty preset to get shure it is not a mistake in your block and mixer configurations. Then check you have this configured in the system configuration: Guitar as Input 1, and Variax as Input 2.

What are the levels then considering these knobs on your headphone?: Volume at 50% and Master at 50%. This should result in a good level for your ears over headphones.

For connecting over the unbalanced 1/4 jacks there exists a little switch (Amp / Line) on the right side of the HD500. Try both positions which is best.

It is absolutely normal to send the signal from the HD500 to your amp with the maximum possible level without clipping the signal. To do this you have to re-adjust the position of the master volume of your HD500 while listening to your amp before the sound gets distort. Because you have a master volume amp you can now lower the level on your amp if it becomes too loud now.

If you have configured the volume by this method, take care that all other presets also reach about this level.

This fact is standing in contrast to your headphones, where you level directly to the exact level for your ears, whereas for amping you level to maximum fitting volume. If you need both at the same time, you have to make your headphones volume adjustable, e.g. by connecting a mixer, or buy a cable with volume control.