When you want to compare two audio samples, it's often hard to tell if you are actually hearing a difference or if your ears and your brain are cheating you. In fact, hearing is a somewhat unreliable sense: our hearing memory is very short and you can lose your bearings very easily...
So, since you can’t trust yourself, you’ll have to make a blind test in order to avoid any preconceptions. And there are many of those in the audio world! Which makes the ABX test a great and simple option: it consists in checking if the subject of the test (i.e. you) can successfully distinguish the difference between quite similar A and B files 10 times or more.
Methodology
1. First of all, you need two audio files (A and B), for example two recordings or two different encodings, that are identical except for one single factor. The difference can be the microphone, the converter or the preamp used for tracking, but also the bit rate or the codec used to encode it. To compare two mic preamps, for instance, use exactly the same audio path except at the preamp stage. To get a reliable test, both files must have the same level, otherwise the loudest one will always sound “better”.
2. To start the test, have the person being tested listen to both A and B files. Every time you play a file to him, tell him which one is A and which one is B, in order for him to have a reference point. The person is allowed to go back to this part at any time during the test. Once you’ve done this you can start the blind test.
3. To do this, let the person listen to one of the two files chosen randomly (this is the X factor) and ask him whether it’s A or B.
4. Repeat step three 10–25 times (but not more, to avoid ear fatigue). Every time, the file you play back must be chosen randomly.
Test results
The audio gear used to play back the sound samples is crucial. That’s why it’s important to keep a consistent audio chain in your home studio: there is no need to buy high-grade converters if your monitor speakers are of poor quality. At the end of the 10 questions, the person gets graded on a 10-point scale. If the grade is 9 or 10, the person’s ability to distinguish both audio samples is confirmed. On the other hand, if you recorded at least two errors, this means the person is not able to distinguish the files. If you ask 25 questions, the test person should answer right at least 18 times.
You’ll find software tools allowing you to make ABX tests pretty easily. For Mac OS X, the simplest solution is to download the free ABXTester on the App Store. The only disadvantage of this particular software is that it only plays back the files five times, so you’ll have to repeat the test at least twice. For Windows, you can use foobar2000 or Amarok. Also check out aveX for Linux and ABX patcher for Max/MSP.
Unfortunately, there is no online ABX test on the Web yet…