Thanks for the reply, and sorry for getting back so late!
I have now made sure the OS sound device is set to builtin microphone, and chosen the Pod Go channels 1&2 as input in GarageBand. Also checked the settings in GarageBand, and they are set as you suggest. Still, experiencing the exact same issue.
For output device I have tried either builtin speakers or bluetooth headphones, but that does not seem to make a difference.
One new finding is that if I move the USB to a different port on the mac, the sound sometimes work for a few seconds right after plugging it in. It then degrades back to beeping noises. So, perhaps this has something to do with the USB drivers on the Mac