One of my favorite parts of teaching middle school boys the past five years is that we spend a quarter of every fall semester slow-watching Into the Woods. In my opinion, it's a perfect musical for this transitional age - a show that is constantly exploring what lies "in-between" the safety and familiarity of childhood/home and the excitement and danger found in wishing for more freedom and responsibility. Every middle schooler lives in this transition between kid and grownup.
In addition to watching, we do multiple projects which give the students a hands-on opportunity to engage with Sondheim's musical ideas. This year, I found a good deal of success getting groups of four students to learn a central section of the big "Your Fault" number in which characters argue back and forth in rapid-fire fashion. (Believe me, rapid-fire argument comes naturally to these boys - but they don't usually do it with a beat.) The students also work on digital audio projects - using a Garageband-like educational platform called Soundtrap - in which they are provided with motifs from the show which they can re-mix by looping, changing instruments, changing tempo, adding beats, etc.
This year, to give them more opportunity to interact with "Your Fault," I entered all the notes in MIDI format for the five characters and the piano reduction of the accompaniment. Once I had this complex two-minute ensemble reduced to data, I knew there was a lot of potential energy for me to do something creative. For starters, I just made this simple re-mix, meant to sound kind of silly and lighthearted. (It's all saxes with a beat, bringing out the playfulness of the back and forth but minimizing the angst.):
Nothing fancy. Though it certainly takes some of the humanity out of the characters, I really like the way it showcases the mechanical ingenuity of Sondheim's restless ensemble. When characters are running around performing this on stage, trying to make lyrics clear, it's almost impossible to achieve rhythmic perfection. That's fine, but I like hearing the argument in pure musical form.
Somewhere along the way, I had the idea that it would be fun to use this data for an animation of the scene, and over the past couple of weeks I've been spending idle hours making my own little "Your Fault" machine using my old friend Scratch. There's a lot more I could do with this (believe me, I have so many ideas), but I've arrived at a very satisfying stage, so am sharing what I have.
The primary concept is to use the MIDI data (basically, information about which notes to play and when) to trigger simple movements for the characters. Since this song is fundamentally about characters pointing at each other and saying "it's your fault," the movements are mostly just about facing and pointing in the right direction, and the characters also bounce nervously in response to each note they "sing." I also added cartoon-like captions with the lyrics. One of the most fun aspects of the project is watching the way the lyrics interact with each other from the four-plus corners of the frame. No matter what the tempo, the movements and lyrics are always cued precisely by the same data which cues the note changes.
Because MIDI is so flexible, I couldn't resist creating the option to re-assign instruments and volume levels for the characters. Unfortunately, Scratch doesn't really have many great sounding instruments, so the choices aren't that great, but among other things, it's easy to effectively mute a character (or the drums or whatever) if you want to try playing a role yourself. Feel free to give it a try here:
Note that if you follow this link, you'll find instructions for how to customize playback.
You may view a demo run-thru and then some demo of setup features, following by a blistering marimba run here:
If you don't mind me patting myself on the back, I'll walk through some of the things I had to accomplish. Scratch is a fun, flexible environment, but not exactly made to do what I was trying to do. Although Scratch objects can be told to play x pitch for y number of beats, it's not naturally designed to run multiple parts at once, and getting the data into Scratch takes some creativity as well. It's not remotely as simple as saying, "Hey, Scratch, play this MIDI file."
As mentioned, I had a head-start in that I already had all of the notes entered as MIDI, using a different program. (A fun cheat here is that, though Sondheim's music is highly chromatic and constantly changing keys, I didn't have to worry about enharmonic spellings. For MIDI purposes, a C-sharp or D-flat will sound the same in this context, so I didn't need to worry about precise note spelling as I hastily played the notes in.) Because I don't really have a good method to have a single part play multiple notes, I did have to spend some time simplifying the accompaniment to be two single-line parts. That was fun, but I didn't knock myself out looking for perfection. I think what I have gets the job done.
Although MIDI data like this is pretty darn simple, I had to use a little command-line tool called "midicsv" to convert from its natural binary state into a text-based file I could manipulate in a spreadsheet. In Excel, I created a few simple functions which allowed me to turn the information for each character into simple lists of pitches and rhythms. Scratch then has a simple way to important such lists into variable sets, so I had the fuel I needed to run the machine.
The rest was basically a matter of creating the cartoonish characters (did not invest a lot of time in that!) and their various poses, identifying which lyrics trigger which movements, and designing an interface for the user to adjust parameters. Again, Scratch doesn't necessarily have this stuff built in, so everything from the volume/tempo sliders to the highlight boxes which show instruments selected had to be created from...well, from scratch. (This usually means writing functions that, for example, keep the slider bar moving horizontally with the mouse, but don't let the vertical position change and don't allow the slider to go past the two endpoints.)
Although MIDI data like this is pretty darn simple, I had to use a little command-line tool called "midicsv" to convert from its natural binary state into a text-based file I could manipulate in a spreadsheet. In Excel, I created a few simple functions which allowed me to turn the information for each character into simple lists of pitches and rhythms. Scratch then has a simple way to important such lists into variable sets, so I had the fuel I needed to run the machine.
The rest was basically a matter of creating the cartoonish characters (did not invest a lot of time in that!) and their various poses, identifying which lyrics trigger which movements, and designing an interface for the user to adjust parameters. Again, Scratch doesn't necessarily have this stuff built in, so everything from the volume/tempo sliders to the highlight boxes which show instruments selected had to be created from...well, from scratch. (This usually means writing functions that, for example, keep the slider bar moving horizontally with the mouse, but don't let the vertical position change and don't allow the slider to go past the two endpoints.)
One thing that makes this different from most Scratch projects I've designed is that I wanted to make this mobile-friendly. The older version of Scratch wouldn't even work on mobile devices, but since 2019, projects can run on mobile, with varying degrees of success. This means that simple keyboard commands don't work very well (those are generally easier to program), but anything which reacts to mouse clicks also works pretty well with finger control on a phone. Since the dialogue boxes Scratch uses to solicit text input are pretty ugly and take me out of the imagination world a bit, I like having all the controls accessible from within the little world I've made.
As I've written before, what really makes all of this satisfying is to experience the ways in which coding are similar to composition. It's always about figuring out how to create a structure based on a web of interrelationships. When the design works, the result can feel like magic even if it's grounded in math. Sometimes this is due to a truly creative, unique solution to a problem (like when a composer writes an amazing melody) and sometimes it's just the satisfaction of using standard formulas effectively (as when a composer uses established principles of harmony and voice-leading).
As I've written before, what really makes all of this satisfying is to experience the ways in which coding are similar to composition. It's always about figuring out how to create a structure based on a web of interrelationships. When the design works, the result can feel like magic even if it's grounded in math. Sometimes this is due to a truly creative, unique solution to a problem (like when a composer writes an amazing melody) and sometimes it's just the satisfaction of using standard formulas effectively (as when a composer uses established principles of harmony and voice-leading).
Of course, there are MUCH more robust languages with which to program that would create higher-quality results all around. But aside from the fact that the childlike Scratch aesthetic suited the subject matter, it also makes me feel more connected to know how the internal mechanics work. Having to think specifically how to create functions which read, react to, and sync MIDI-like numbers is a more hands-on experience than using pre-programmed plug-in functions - just as it's more satisfying to create a musical composition by choosing all the pitches than it might be to use pre-fabricated music loops. This program will definitely glitch sometimes, especially if you push the tempo up over 150 (hint, hint), but it's remarkable to me how well it holds up considering all the decision-trees I know it's processing and how many notes have to be triggered in such little time.
I now have a pretty cool set of music projects created in Scratch. This one is most similar in scope and function to "Poppy Bach," which plays a three-voice Bach fugue and gives the user a wide variety of customization and view options. But I'd encourage you to check out each little scratch-world found in this gallery.
While I'm here, it's also worth pointing out another "Your Fault" project I created within the class context a few years ago. In this case, I had done some editing of the audio from the filmed 1987 Broadway production, so that the music stays reasonably close to an unchanging beat, which means I could easily drop in other beats and sound effects. Although I should probably re-edit to turn down the drums a little, I think this is pretty fun!
As an epilogue, I'll note that the great musical humorist Peter Schickele (inventor of PDQ Bach, etc.) passed away recently. As I've thought about his legacy, I thought about how much he and Victor Borge and Igudesman and Joo and Two Set Violin all benefit from the generally ultra-serious attitude with which those in the classical music world think of "playing" music (even music intended to be lighthearted). All of these comedians have become popular and established enough that their audiences take great delight in the irreverent treatment they display towards the canon, but I continue to wish we cultivated a more genuinely playful attitude in general.
I now have a pretty cool set of music projects created in Scratch. This one is most similar in scope and function to "Poppy Bach," which plays a three-voice Bach fugue and gives the user a wide variety of customization and view options. But I'd encourage you to check out each little scratch-world found in this gallery.
While I'm here, it's also worth pointing out another "Your Fault" project I created within the class context a few years ago. In this case, I had done some editing of the audio from the filmed 1987 Broadway production, so that the music stays reasonably close to an unchanging beat, which means I could easily drop in other beats and sound effects. Although I should probably re-edit to turn down the drums a little, I think this is pretty fun!
As an epilogue, I'll note that the great musical humorist Peter Schickele (inventor of PDQ Bach, etc.) passed away recently. As I've thought about his legacy, I thought about how much he and Victor Borge and Igudesman and Joo and Two Set Violin all benefit from the generally ultra-serious attitude with which those in the classical music world think of "playing" music (even music intended to be lighthearted). All of these comedians have become popular and established enough that their audiences take great delight in the irreverent treatment they display towards the canon, but I continue to wish we cultivated a more genuinely playful attitude in general.
There's a lot of mistrust about what happens when musical ideas are reduced to math. While I agree that something is lost when we take singing actors out of a number like this, I think this process enable us to appreciate something about the craftsmanship beneath the surface. (Also, again, this program will stumble every now and then so it's not so mathematically pure due to my own human frailties as a programmer! Take that, AI.)
No comments:
Post a Comment