If you are confident that you are tapping at just the right time then you could try:
1) making sure that your USB cable is unplugged while you are looping.
2) As a last resort: try a global settings reset using the procedure here ==> http://line6.com/support/docs/DOC-2222 . This can sometimes clear up weird glitches.
There is no metronome or quantisation with the HD looper AFAIK so it can be hard to get it really sharp.
+1, The looper can take a little getting used to. You should only use it when NOT connected to HD Edit as this can cause some latency in the device. This is normal when connected to a computer because of the time it takes for the 500 to communicate with your PC/Mac. Otherwise, try counting the beat off (possibly with a separate metronome or drum loop). If the loop is off beat you may hear a gap.