Using Tone Transformer to Create Avant Garde Jazz

I present to you Carol by the Pepe Silvia Trio ft. David Dockery – enjoy!

It’s definitely a bit too “modern” for my tastes, but it has some appeal to it. I particularly like the bit of synchronization at 1:15, the band is really locked in there. Has some Yo-Yo Ma & Silk Road Ensemble vibes.

So what exactly is this noise? If the name didn’t give it away, first you need to watch this:

Tone Transformer (Formerly Timbre Transformer)

Yesterday I came across this fantastic Andrew Huang video where he dives into the Google Magenta Project Tone Transformer. This AI-based program takes a .mp3 or .wav input and alters the waveform so that it sounds like a brand new instrument. As a quick Physics crash course, the timbre of a wave is the “texture” of the sound, all the little bumps that make a violin sound distinct from a trumpet even if they are playing the same pitch.

Using the Timbre Transformer Colab (since it could deal with longer audio files) I ran the It’s Always Sunny in Philadelphia clip’s audio through a few times, creating a trumpet, flute, and tuned violin version of the scene to mix together. You can hear the individual tracks on the side.

Making “Music”

I might be a bit biased, but I felt like the trio needed some percussion to hold things together. Fortunately enough, David Dockery already made a drum track that goes perfectly with the clip!

At this point all I had to do was throw all the tracks together in Audacity and export it.