I am using a 5150ii head into a 5150 cab. I found the below info online around my FX loop. I'm not using a footswitch (since I got the amp used and it didn't come with one) but the way I read that last sentence below, I shouldn't have to ("If footswitch is used, "Effects" must be selected (LED illuminated) for effects to work"). Also, I have the Helix set to instrument for input and send and receive in the global settings.
EFFECTS SEND/EFFECTS RETURN Signals are supplied to outboard effects or signal processing units by patching from the Effects Send (7) output into the outboard unit(s) and back into the Effects Return (8) input using shielded cables with 1/4" mono phone plugs. Only non-gain effects devices (chorus, reverb, delay, etc.) should be used in the effects loop. If footswitch is used, "Effects" must be selected (LED illuminated) for effects to work.
EFFECTS SEND: Load Impedance: 47 k ohms or greater Nominal Output: -10 dBV, 300 mV RMS
EFFECTS RETURN: Impedance: Very High Z, 470 k ohms Designed Level: -10 dBV, 300 mV RMS