That can happen when there’s a mismatch between the sample rate your audio device expects and what it receives. One way to fix this is to force the system to only allow one sample rate. I forget which files need to be edited for this, perhaps someone else will know, but you have a list of accepted and fallback sample rates, and you need to delete all except one.

I can’t say that it will solve your specific issue, but it solved mine and I had the same symptoms.