Ever since the creation of the first DAW, no other software has caused so much ink to be spilled and generated such expectations. The Direct Note Access technology, which was introduced by Celemony at Musikmesse 2008, is one of those holy grails no one ever thought to be accessible because it allowed you to edit single notes of a polyphonic audio recording. Is it some sort of de-mixing? Yes and no! Is it a revolution? You bet!
Ever since the creation of the first DAW, no other software has caused so much ink to be spilled and generated such expectations. The Direct Note Access technology, which was introduced by Celemony at Musikmesse 2008, is one of those holy grails no one ever thought to be accessible because it allows you to edit single notes of a polyphonic audio recording. Is it some sort of de-mixing? Yes and no! Is it a revolution? You bet!
Before we dive into the innards of the program, a brief summary about Melodyne is necessary for those of you who don’t know it yet. Celemony created Melodyne in the wake of the Antares Autotune, which allowed you to edit the pitch of an audio recording. Melodyne worked under the same principles (pitch shifting and time stretching with formant control) within an interface conceived for musicians instead of sound engineers. After detecting the notes, you had several tools for pitch, time and amplitude correction, so you could actually edit audio recordings as easily as MIDI parts, under one condition: the audio recording had to be monophonic. The software’s excellent algorithms and idiot-proof user interface gave lots of product ideas to their partners (like Ueberschall, who developed customizable loop banks for the Melodyne engine) and competitors. For instance, Autotune got a new user interface (see the Autotune EVO), several competitors appeared (Waves Tune, Zplane) and the main audio sequencers integrated Melodyne-like functions (Steinberg introduced VariAudio in Cubase 5 and Cakewalk did the same with AudioSnap for Sonar).While competitors were still trying to catch up with the first Melodyne, Celemony changed the game again by offering individual note editing in polyphonic recordings. During the product presentation at Musikmesse, Melodyne’s boss had a blast changing a guitar minor chord into a major chord using a simple MIDI keyboard. And to top that, he also modified the trumpet of a Miles Davis recording without changing the double-bass or the drum part. Impressed? There are no words to express it. The presentation of the product had such an impact that some people thought it was a hoax. That, together with the time it took for the official release to come out raised serious doubts among the audio community. But…
Melodyne Editor, the first software using Direct Note Access technology (DNA) has finally hit the stores. And it works…
On Familiar Ground
The installation is extremely easy. You will only need the serial number to activate it online on Celemony’s website. The software is protected in two different ways: either you activate the product online, in which case the registration is limited to only two computers (you’ll have to uninstall it first before installing it on yet another computer) or you transfer your license to an iLok key. Once you did that you can start your sequencer (I work with Cubase) and look for Melodyne Editor in your plugin list.
Users of previous Melodyne versions, especially those who had the plugin version, won’t feel too estranged at first sight. The user interface (the look and layout) didn’t change much. Under the Settings, Edit, Algorithm, View, and Help menus, you’ll still find the aluminum-like bar hosting the basic parameters. Most of the interface is made up of a sort of piano-roll grid displaying yellow, orange and red events… On the right side, you’ll still find the “Correct Pitch” and “Quantize Time” buttons, as well as three automation-capable controllers that allow you to play with the pitch, the formant or the volume parameters in real time. On the center of the tool bar you’ll find the Undo/Redo icons and the tool box (with the same old icons): from left to right, you’ll find six tools for selection/zoom/scroll, pitch editing (with modulation and drift parameters – a sort of audio pitch bend), formant editing, volume editing, timing editing and note separation editing.Right below these icons, there are two fields that display the note detected in the segment selected and its distance to the correct note. Finally, on the left side of the bar you have the transfer parameters. Just like with the first plugin version, the first thing you have to do is start the detection process: once Melodyne is inserted in the track that is to be processed, click on the transfer button and start playback in the sequencer. Depending on the algorithm you selected in the “Algorithm” menu, Melodyne analyzes the audio material and generates events on the grid. There are three algorithms available: monophonic (melodic), rhythmic/unpitched and polyphonic. In this review, we will focus on the latter since the two others are already known from the Melodyne plugin.
Before we get into details, we have to mention that, unlike the first Melodyne plugin, you can fully resize the program window and freely zoom in/out via shortcuts. It would have been perfect if it had a button to switch into full-screen mode with a single click…
These Germans Mean Serious Business
With the pitch editing tool, I can click on a segment to listen to it by itself and drag it with the mouse to transpose it, so I grab the whole C line and drag it to C sharp to get an A major chord.
Original sample:
A major chord:
Stunning, isn’t it? Some of you might have noticed that the software only detects four different notes, even though an A minor chord on a guitar has five notes (A, E, A, C, E). The Direct Note Access seems to have difficulties to distinguish notes with the same harmonic content, like the same note in different octaves. If we were to consider the A line in the previous file, you could clearly hear that the segment contains the low and high A notes of the chord:
It’s true that it is a limitation, but the software is still very effective, especially to add ornaments. Here is the same loop played faster and with some ornaments:
It also works quite good with loops containing clearly separated notes. The following example makes the point clearer; in spite of the reverb, which could make detection more difficult, the software does a wonderful job (notice that it allows you to hide or display the decay of the notes, in other words, the reverb effect).
Original sample:
Same sample after DNA:
Not really. I changed the files! Got you, didn’t I? I did it to prove that the algorithm is sophisticated…
Actually, when the sounds are well separated, the software is pretty effective: we had the opportunity to mess around with strings, guitar, piano, and banjo loops and always got convincing results. You can listen to what we did, just for fun, to Oasis’ Wonderwall and The Beatles’ Eleanor Rigby using the DNA…
Oasis:
Eleanor:
On the other hand, it becomes more difficult with other type of sounds: distorted electric guitars have so many harmonics that you get a multitude of tiny audio segments and it’s impossible to know what they represent. It will be much harder to edit such sounds in spite of the intelligent selection tools provided by the software.With a simple mouse action you can select all identical notes, all the same notes in different octaves, all higher or lower fifths, and all notes on the same beat in all other bars. Still, the good old monophonic mode remains very effective to edit a power chord and Melodyne has made that very clear.
In spite of these limitations, the software proves itself to be quite effective: you can create many useful variations from a single loop and correct minor details, like bad tuning during a recording, for example…
All the more since you can cut, copy, paste, or delete audio segments. In most cases, it’s also possible to add a note one octave higher or lower to a brass part to enrich the harmony, but also to delete an instrument either partially or totally. Listen to this sample:
Original sample:
Same sample with some copy-pasted notes added:
The software also provides lots of solutions when it comes to rhythm (quantize, etc.). In this regard, Melody makes wonders in monophonic mode, but with polyphonic signals the results are less convincing and I couldn’t really obtain satisfying sound examples. I guess it ought to be considered on a case-by-case basis.
Also notice that you can help the software in the note detection process using the dedicated tool, especially when it comes to separating notes. Pitch detection will be so much easier…
Polyphony
That’s all very nice, you say, but the ultimate application of the DNA wonder is de-mixing, right? Celemony claims it never intended this to be the main aim of the application, though it is interesting to try it on full mixes. Considering that it is possible to delete audio or edit one note without changing the others, the question would be if it is possible to modify the guitar solo in Stairway to Heaven or to extract the mythical bass line of Billie Jean.
Melodyne can be stunning but also disappointing in this respect, perhaps because this is what we hope to see in the future. The software is more or less effective depending on the audio material you use, and there is no way to know what the DNA will achieve before actually trying it out. On Chet Baker’s Autumn Leaves, it is possible to edit trumpet and bass lines independently even if some artifacts appear in the drum part’s upper frequency range.
Autumn leaves:
And the revisited version (Spring leaves?):
It’s the same with Lou Reed’s Walk On The Wild Side: the intro part, with its light rhythm guitar and famous double-bass playing octaves, is a delight for Melodyne. You can enjoy yourself changing the melody:
… or deleting the higher octave note, keeping only the lower notes of the double-bass:
It’s true that, in both examples, small artifacts appear in the guitar’s upper frequency range, but mixed with a fat hip-hop beat, nobody will be able to hear how you did it.
On the other hand, with Julia (The Beatles), you won’t be able to separate guitar and vocal parts, so you will be limited to doing some sound design. Actually, as we had already noticed with octaves, the software strains to distinguish notes having almost the same harmonic content. What’s more, even though Melodyne is now polyphonic, it cannot manage multitimbral audio material: if a sax and a cello play the same note, Melodyne cannot separate the two instruments and will only deliver a single audio segment for both of them.
It occurred to us that a song mixed with extreme pan settings, like The Beatles’ I’m Looking Through, would give better results, but Melodyne wasn’t convincing enough because you cannot edit each channel of the stereo signal separately. It is still a long way to go for the de-mixing Grail, even if no other tool can go as far as Melodyne. With the Beatles’ song, you can try to edit the left channel first and then the right channel…Maybe the result will be more convincing.
With a complex mix like Air’s Electric Performers, you won’t be able to edit the guitar arpeggios, but you will achieve very interesting sound textures editing time and pitch parameters. Celemony’s technology could almost be a totally new type of sound synthesis…
Conclusion
Melodyne Editor is indeed the revolution we expected, thanks to its DNA technology. The algorithm is not infallible and (still?) doesn’t allow to entirely de-mix a song, though it was probably never intended for full mixes. Nevertheless, there has not been such an exciting invention in the audio industry ever since the creation of Autotune – and the invention of samplers before that. Melodyne is available for a very affordable price considering the huge R&D efforts Celemony had to make to achieve these results.
We strongly recommend Melodyne Editor to sound engineers (to repair an acoustic guitar recording when the guitarist already left the studio), to musicians who work with loops (and never find the right sample in the right key) and to all sound designers. You will undoubtedly have lots of fun discovering the huge possibilities it provides. However, there is still one thing that remains unclear: what happens to the copyright of the processed samples? If I change all the notes of a Miles Davis phrase, will he still be the owner of the melody I use in my song?
To wrap it up, if I were to have only one gift under the Christmas tree, I’d ask for this one!