Wednesday, April 16, 2014


For years now, I've made much of the fact that the MM* in MMmusing can stand not only for my initials, but also for "Music and Meaning" as well as for "MultiMedia" or "Multimedia Musing" - I hadn't envisioned that multimedia creations would be so central to this blog when I chose the name, but perhaps the name chose me. Anyway, up until now, multimedia has most often meant: video, audio, Photoshopped images, and even some bits of javascripting to connect videos, scores, and analysis, etc. But today, I present my first real "game" entry among these multimedia offerings. Here's a little demo:

[Play Solfèd for real here.]

I've never studied computer programming, although I've often thought if I'd started college 10 years later than I did (when multimedia integration picked up pace), I might have ended up in that world. I have dabbled, off and on, with coding at various points, first when I programmed a beloved Hewlett-Packard calculator to play a version of Mastermind. I was always particularly proud of the fact that that little program used the absolute maximum number of steps (about 320?) allowed by memory capacity; although there was probably a more efficient design possible, it seemed as if I'd wrung out all of the machine's "mastermind" potential through creative problem-solving - kind of like composing the perfect 140-character tweet that seemingly wouldn't have been possible with 139. Or writing a perfect piece of music for one bow and four strings.

The point here is that programming can be a very artistic kind of design process, finding unexpected potential within seemingly rigid constraints - certainly different from musical composition, but related in some tangential ways. A big difference is that when notating music for human performers, one can usually count on the human to intuit to some degree what one is requesting; a computer program, for better or for worse, does exactly what you tell it do to. (I know most composers think that's what they want from performers, but they might be surprised if they really got what they asked for...) There's tons of problem-solving and laborious bug-detecting to be managed, but when a bunch of instructions suddenly produce an elegant result, it's immensely gratifying and genuinely beautiful. 

My next foray into programming after mastering the calculator involved using Hypercard's scripting language on a very early Mac (late 80's) to design a page that drew fractals. (This was for a class presentation on fractals and musical design.) The machine and my program were so slow that it would take hours for the fractal to emerge from all the intricate iterations of whatever function was being plotted (I don't remember much about the details) - and, of course, the fact that it took hours was part of what made the process beautiful. I wish I had access to those programs now, but they basically generated images like this by graphing out a series of calculations. It seemed like musical magic, and working out those fairly basic instructions was very satisfying. 

Then along came the Internet, and I've since spent more time faking my way through video/audio-editing and web design than any real programming. UNTIL, a few weeks ago, my screen-mad 6-year old son asked about downloading an iPad app that allows you to create apps. He loves playing with the iPad and is constantly searching for new games ("Dad, this one's free, it got 5 stars, and it's for ages 4 and up!"), which can be a bit tiring, but I loved the idea that he wanted to create his own games. The app he'd found didn't seem very kid-friendly (although I'll definitely be checking it out), but I remembered that my older daughter's 6th grade class had done some basic work in MIT's free Scratch programming language a few years back. It's designed to be very inviting and kid-friendly, so we dove in.

Within a few days, we (OK, mostly I, though Son of MMmusing proved quite good at catching little errors I'd make along the way) had come up with our first game - a variation on the old Pong classic called Pinguin-Pong. I think it's fun and colorful and appealing on a simple level and am most proud of the fact that the flexible leveling for each player makes it easy to set up a good competitive game between two players of different abilities. (Yes, I can still beat my son when even-up at video games, but that's not likely to last...)

The main point is that I found Scratch to be insanely fun and addictive - one can certainly still make plenty of syntactical errors along the way, but the graphical interface and intuitive design make the idea of "building instructions" much easier to visualize and conceptualize than coding systems I've dabbled with in the past. Naturally, my mind turned to the idea of musical games and, since the platform already includes some basic MIDI-like musical software integration, I decided it would be fun to set up an arcade-style game to test pitch-recognition skills. I know I'm far from the first to design such a game, but I was more interested in learning my way around the platform than in breaking new ground. Sometimes, doing something with limited ability is more fascinating than doing something one is trained to do. (See/hear my "viola" playing.)

I originally had in mind a sort of Space Invaders game, but I wasn't ready to deal with multiple aliens at once, so it was going to be a "shoot the ship from the sky" kind of thing. Somehow, the ship became a bird, and then I found it a bit disturbing to see these merrily chirping birds shot through with red darts (you can see an early draft here), so, sensitive new-age dad that I am, the concept evolved into a bird-feeding game. I experimented a bit with using more of a bird-tweet sound for the pitches, but that was more annoying than it was worth. So, I settled on a nice vibraphone vibe that doesn't really sound like a birdcall, but it's mellow enough that I don't mind hearing this virtual instrument over and over. (I wouldn't want to put a number on how many pitches I've heard this vibraphone vibrate in the past week.)

The finished-for-now project turned out, not surprisingly, to be much more complicated than I'd expected, even though the result is pretty simple in appearance. I can't pretend it's completely designed "from scratch" as the Scratch platform has a lot of built-in multimedia capability, but, for example, all of the pitches and their placements on the staff had to be programmed (Scratch doesn't know about music notation, which involves lots of little irregularities that have to be accounted for), as did the mechanism for sending worms into the birds and for sending "bad worms" offscreen. Actually, the worm-rejecting is one of the weakest graphical features at this point, but I'm not in a big hurry to change anything. YOU can actually go right into Scratch and modify the program any way you want (doesn't change the posted version), or at least look around to see what's under the hood.

[Feel free to skip this clunky paragraph!] Here's a quick look at what one script looks like - this script (which is recalled many times consecutively as each new note is created) recalls pitches that have already appeared and plots them out in the correct horizontal/vertical spots on the staff. Amazing how vexing something like a ledger line can be - or how one suddenly realizes a barline means subsequent pitches need to shift over a bit. Also, I had decided I liked the idea of the pitches being plotted according to their pitch value, as opposed to me simply telling the program where each individual pitch belongs, so that higher pitches would automatically climb the staff. Because a diatonic scale means some notes that look equally spaced are a half-step apart and some a whole-step apart, this required some extra work to be sure the half-steps traveled far enough..

Because I'm self-taught and the kind of guy whose office usually looks like something exploded, my coding is surely not the most elegant (I just noticed a few "wait for 0 seconds" commands there that are kind of the equivalent of old coffee cups collecting dust on my shelves), and I'm sure there are simpler solutions to many problems than what I devised - as it is, there are more than 150 scripts and more than 50 variables floating around in there, all in the name of feeding virtual birds. But, fledgling programmer that I am, I can still say that it is a joyful and beautiful thing when, after hours of logical problems, a simple button click makes all the right things happen.

Again, I know there's plenty of ear-training software out there, much of it much more robust and flexible than what this game has to offer. For example, the much-used has a bunch of resources, including this note-ID exercise that allows for multiple octaves, chromatics, etc. However, I do feel that my little birdfeeder feels more like a game - it helps a lot that one can use keyboard input to play Solfèd. If speed is your thing, it's much faster and more viscerally gratifying to type in the worms than it is to move a mouse around the screen. One thing I like about Solfèd's set-up is that, since there are only 8 pitches, one can easily rest 8 fingers across the number keys and work on speed reflexes without moving the hands. Gives it more of the Guitar Hero kind of thrill, except this depends more on the ear than the eye. As I've observed many times before, the hand/eye coordination involved in sightreading at the piano is basically like playing a video game - this just switches the demands over to the ear, which is good training for me as my eyes are much better "readers" than my ears.

Although I have to admit my son wasn't very closely involved in designing this game, he and his 9-year old sister have been very willing to try it out, and of course I don't mind if they develop their hearing skills along the way. To be honest, my own ear-training was never what it should've been, and I still find it helpful (and fun) to try to speed through this game, relying as much on instinctive reaction as I can. I don't have anything close to perfect pitch and am OK with that, but I tend to hear so functionally, that I have to really fight myself not to always hear each pitch relationally** - not that there's anything wrong with that, but I'd like to see if my fingers can eventually hear a pitch and fire without thinking about where the pitch is headed...

The good thing is that, unlike reading this post, playing the game can take just a few seconds. It's very easy to figure out, and you can set yourself whatever sort of time/mistake challenges you like. It's also fun, after playing a few rounds, to set the playback speed to something insane like 300 and hear the vibes fly. Go feed the birds!

* Bonus MM: When I first started this blog, I was living in Medford, Massachusetts.

** For example, when I hear "Fa," my typical brain process is to hear it leading down to "Mi," and it's only when my brain hears the "Mi" that the "Fa" is confirmed - which takes up valuable micro-seconds when you're trying to feed a hungry bird.

*** 2023 Update: I've also made a version of this game which shifts DO each time you play. (The pitches shown are still in C Major, because reacting to various key signatures would've been a LOT of work.) GO HERE.

No comments: