AU is strictly a Mac format, whereas VST works on both Mac and PC. Most of the time, the choice of plug-in format is dictated by the DAW you use, although some DAWs, like Ableton Live, can handle both AU and VST. What's most important is the quality of the plug-in, rather than its format. On the Mac side, most plug-ins are available in multiple formats, and you can chose which one (or ones) to install.