That's quite unusual.
The 18i20 is effectively a plug and play device on Mac and we're not aware of any issues with Scarlett and Yosemite.
Try also deleting the following:
Mac HD -> System -> Library -> Extensions -> Delete FocusriteUSB2Audio.kext, FocusriteUSBAudio.kext & NovationUSBAudio.kext if they are present.
Once done, please navigate to:
Finder -> Go Menu (top of screen) -> hold 'ALT' on keyboard -> select newly appeared 'Library' option -> Preferences -> delete com.apple.audio.audiomidisetup.plist
Once done, navigate to:
Mac HD -> Library -> Preferences -> Audio -> delete both files.
Empty the trash and then restart your computer
Do you get the same issues when using the 18i20 across all the USB ports on your Mac? Try unplugging any other non-essential USB devices to see if this helps across each USB port.
Are you using the original USB cable? If so, try using a different USB cable that it under 1.5meters.
It would also be helpful to see if the same issue occurs if the unit is used with another computer.
Let me know how you get on.
Simon // Focusrite Technical Support