The battery isn't the problem since it only starts when the battery is fully installed. Maybe the variax circuit bleeds to the output when it's not on or something. It should be completely off but it isn't since it affects the sound with noise.
Another lesson here, should have bought it from the local shop.