First of, let me point you to this excellent read:
USB (just like magnets) are surrounded by this “magic” (which is just engineering/science in my eyes).
So while you can have parasidic noise on the USB power, ANY device that relies on that power being clean should bring its own filtering.
There are proper power filters/conditioners for low voltage (example below), there are USB isolators (mostly for lab/industrial use when probing high voltage equipment) those cost a lot Example
Audiophile USB-stuff
This here is a Audioquest Jitterbug without the case:
There are two inductors, a diode (why?) and 3 capacitors in paralel with a high resistance resistor. Total parts cost is less than 3€ (with a nice PCB, maybe 6 when building only 100).
Those inductors (= chokes) filter some noise. For a proper filtering on the power rails, you want what is called “common mode choke”.
Working principle:
And here are some common mode chokes in the palmer purifier (blue arrows):
The Wyred Recovery lacks filtering completly (not even any meaningful capacitors in place):
The Ideon 3R Master Time
That is one hell of a big powersupply (why? Going to charge your tablet of it?). The torroidal transformer provides filtering against grid noise. Has a nice bunch of capacitors in there.
Could not find any pictures of the iFi dongle, so if anyone here owns one and can provide me with some pictures of the insides 
There are noisy USB power sources. There is equipment that does not bring its own filtering to the table. So there is a valid reason to plug stuff into a powered USB hub or use a filter on the power. Leave the data alone!
Someone in China figured out how to do this:
Capacitors accross the power and Gnd rail, common mode choke and an LED with appropriate resistor to show the device has power. Costs $8 shipped.
Want some solder practice?
Digikey parts:
Edit: Messed up, one of those Mini-B breakout boards needs to be a USB-A socket. Or get a cable like this