Given you had a similar problem with the onboard sound I wouldn't blame the usb adc. I'm inclined to think the problem is em interference, which could be from just about anything; cellphone, wireless router or mouse, monitor display...anything that uses an smps, light dimmers, fluorescent fixtures, etc..
The cable that connects the audio to the pc is possibly acting as an antenna, and this may be true even for the usb cable. If you're using inexpensive ready made cables for audio, these often have poor shielding. It might be worth a shot to make up a set using 75 ohm quad shield coax, though it would be tricky to carry the shield into the mini plug. Every plug would have to be jacketed in metal to ensure the shield is complete from end to end.
Also, try to position the sound card in the lowest slot, furthest away from the other boards. Every bit helps.