This weekend I’ve been playing, once again, with the ideas of experimental film pioneer John Whitney, using both graphics and audio. While Whitney was interested in turning musical ideas into motion graphics, I’m doing the inverse — turning one of his key animation ideas back into music.

Whitney made a number of films based around the simple idea of harmonic relationships. Above is a visual example of one his ideas that I implemented in Flash.

Here’s Whitney talking about this idea:

An early intuition about how to control total dynamics led me to activate all graphic elements through a motion function that advances each element differentially. For example, if one element were set to move at a given rate, the next element might be moved at two times that rate. Then the third would move at three times that rate and so on. Each element would move at a different rate and in a different direction within the field of action. So long as all elements obey a rule of direction and rate, and none drifts aimlessly or randomly, then pattern configurations form and reform. This is harmonic resonance, and it echoes musical harmony, stated in explicit terms. I tried this procedure in several films, and was gratified by the consistency of the confirmation it demonstrated.
— John Whitney, “Digital Harmony“, pp. 38

In this movie, each of the 48 dots is moving in a circle. Each of the dots is on a 3 minute cycle. At the end of 3 minutes, the outermost dot will have moved around the circle once (this dot represents the first harmonic or fundamental). The next dot will have moved around the circle twice (representing the second harmonic). The next dot three times, and so on. The innermost dot moves around the circle 48 times.

So, I had a cool idea: Imagine these dots are raised bumps on a disc which is controlling a music box, with each bump triggering a note when it passes the zero degree line (a line extending from the center to the east). The result would look and sound something like this three minute looping movie, which I built in Flash:

Whitney Music Box (chromatic version)

In this particular implementation, the music box has the bars for a 48-note chromatic scale. The pattern causes chords, ascending and descending runs and melodies to be heard.

You may notice some interesting links between the visuals and the audio, especially if you are a musician. For example, when the pattern forms a 3-arm starfish, the chords you are hearing are diminished chords, which consist of minor thirds, an interval in which the notes are 3 chromatic steps apart. The chords you hear always bear this type of relationship to the pattern you are seeing, consisting of intervals which match the arrangement of arms.

I generated the audio using my sound synthesis program, Syd, which provides a very elegant way to accomplish this kind of thing. Unfortunately, Syd is horribly supported these days (I haven’t updated the Macintosh version since before OS X). Here’s a sample Syd patch which was used to generate the audio (I actually used 3 similar patches and mixed the results together).

Update:

Here’s are some other variations. The first few assign pitches based on the harmonic series, rather than the chromatic scale (thus the dot that represents the 10th harmonic actually sounds like the 10th harmonic). In these variations, the “chords” sound like single notes with a buzzier timbre, because when you combine harmonics, you are performing a kind of additive synthesis.

Whitney Music Box (harmonic version – fundamental = 55hz)

Whitney Music Box (harmonic version – fundamental = 20hz)

The above variation uses a lower fundamental which is close to the limit of human pitch perception.

Whitney Music Box (harmonic version – fundamental = 48hz, reversed)

The above variation reverses the assignment of the pitches, so that the lower fundamentals are heard more frequently.

Whitney Music Box (harmonic version – Palindrome)

The above variation uses both forward and reverse pitch mappings at the same time, to create a palindrome effect. The oscillators with the reverse pitch mapping are 180 degrees out of phase, which causes some nodes to cancel each other out, and creates some mildly interesting filtering effects.

Whitney Music Box (chromatic version – 88 tines)

The above variation uses 88 tines in a chromatic arrangement with alternating ascending and descending notes. Because there are more bumps on the wheel, there are more notes in the sequence.

Update 4/25/06:

I’ve generated a few more variations, which you can see here. I’ve begun using STK (the synthesis toolkit) instead of Syd to generate the audio, since this is more commonly available software that will allow other folks to experiment with this type of system. I generate the audio score using a Perl script which outputs note events in the SKINI format. I use the command-line utility calico to render the audio.

Update 4/26/06:

Nyquist, the Lisp-based music language, also provides an elegant way to generate this type of music. Here’s a sample nyquist score that I used to generate variation 11.

Update 5/2/06:

https://blog.krazydad.com/2006/05/02/a-fractal/In this new post, I describe the nature of the star-fish patterns in more detail.

Update 9/22/11:

I added links to Whitney’s book in the blog post, and slightly modified the text for clarity. I changed the original Flash illustration to a GIF, so it can be viewed on IOS devices. I’ve updated the links to point to their current locations. You’ll find lots of sample code on Github.

This entry was posted on Sunday, April 23rd, 2006 at 4:25 pm and is filed under Jim's Projects. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.