Melodies of Life
I played an unhealthy amount of video games in the late 90’s/early 00’s, especially JRPGs. Summer breaks would consist of long binges of Final Fantasy 8 and 9, Chrono Cross, and other gems like Brave Fencer Musashi and Tomba! With all the time at home I have been trying to revisit some of these old games, and I can say with confidence that the memories of them have aged far better than the games did. One thing that did withstand the test of time has been the music of these games, though that is mostly driven by personal nostalgia since the audio quality of these old files can be a bit lacking. This low file size can be a great advantage for playing around with different programs to enhance or learn from the music, so that’s exactly what I set out to do!
Specifically I wanted to target the issues with how short and repetitive these themes can be. A lot of these pieces are fully developed, but because of the way they are experienced in game at most maybe twenty or so measures are regularly heard. To that end, I found two programs that would learn from a short snippet of sound and then work with that to extend it or build complexity with an accompaniment.
Compression of Time
To get this all to work, I had to get some clean files to work with. For this project I needed MIDI files, ideally type 0. A quick search led me to some MIDI covers of the pieces I wanted, but these needed to be cleaned up to work. I started by using MuseScore to trim the files and eliminate any distracting instrumentation like excessive percussion or unbalanced dynamics. From there I had to run it through a converter I found online to bring it to type 0 MIDI. This led to some files losing a lot of their musicality, but I still had a handful of workable music.
MuseNet
The first tool I used was OpenAI’s MuseNet. This program was unique as it actually just runs in a browser as a widget on their blog. This limits what I could do with it but it still had some solid results. I ran the same clip of Freya’s Theme from Final Fantasy 9 through it and got a couple different results:
Freya’s Theme, MuseNet Extended v1
Freya’s Theme, MuseNet Extended v2
What I found most interesting about how this program works was how much it relied on momentum rather than memory. It would build each moment off of the previous one using the library of sounds that trained it, leading to some odd artistic choices like the percussion that takes over the second clip.
Music Transformer
The next program I dove into was Magenta’s Music Transformer. By using a Colab they built (with a minor fix to make sure the output and the original files could properly join) I was able to quickly make extended versions of some pieces, along with a visualization of the file. You could run the same clip and get drastically different results, for example, here is the Chocobo theme with three distinct takes:
Chocobo Theme, FF8, Music Transformer v1
Chocobo Theme, FF8, Music Transformer v2
Chocobo Theme, FF8, Music Transformer v3
Though it is generally pretty obvious when the original file ended and the extension began, the transition is not too abrupt. The use of MIDI files helped make sure things like key and tempo carried over, though it was clear at times that there were some creative liberties the model chose to take. Here are a few more samples I ran through the program:
A Place to Call Home, FF9, Music Transformer
Freya’s Theme, FF9, Music Transformer
Chocobo Theme (Extended Sample), FF8, Music Transformer
Peril, Chrono Cross, Music Transformer
Twinpeak Mountain, Brave Fencer Musashi, Music Transformer
Music Transformer can also build accompaniments to pieces rather than continuations. These files strip the MIDI down even further, only keeping a single tone at a time, but then creates some fairly rich results. For example, below is the stripped down version of Dances with Lizards from Chrono Cross and the accompanied version it generated:
Dances with Lizards, Chrono Cross, Piano Redux
Dances with Lizards, Chrono Cross, with Accompaniment
It kind of reminds me of some of the crazier breaks in Rhapsody in Blue. I ran a couple more pieces through the accompaniment generator and got the following, with visualizations:
Blue Fields, FF8, with Accompaniment
