Blog  |   Puzzles  |   Books  |   About

Visual Harmony

This weekend I’ve been playing, once again, with the ideas of experimental film pioneer John Whitney, using both graphics and audio. While Whitney was interested in turning musical ideas into motion graphics, I’m doing the inverse — turning one of his key animation ideas back into music.

Whitney made a number of films based around the simple idea of harmonic relationships. Above is a visual example of one his ideas that I implemented in Flash.

Here’s Whitney talking about this idea:

An early intuition about how to control total dynamics led me to activate all graphic elements through a motion function that advances each element differentially. For example, if one element were set to move at a given rate, the next element might be moved at two times that rate. Then the third would move at three times that rate and so on. Each element would move at a different rate and in a different direction within the field of action. So long as all elements obey a rule of direction and rate, and none drifts aimlessly or randomly, then pattern configurations form and reform. This is harmonic resonance, and it echoes musical harmony, stated in explicit terms. I tried this procedure in several films, and was gratified by the consistency of the confirmation it demonstrated.
— John Whitney, “Digital Harmony“, pp. 38

In this movie, each of the 48 dots is moving in a circle. Each of the dots is on a 3 minute cycle. At the end of 3 minutes, the outermost dot will have moved around the circle once (this dot represents the first harmonic or fundamental). The next dot will have moved around the circle twice (representing the second harmonic). The next dot three times, and so on. The innermost dot moves around the circle 48 times.

So, I had a cool idea: Imagine these dots are raised bumps on a disc which is controlling a music box, with each bump triggering a note when it passes the zero degree line (a line extending from the center to the east). The result would look and sound something like this three minute looping movie, which I built in Flash:

Whitney Music Box (chromatic version)

In this particular implementation, the music box has the bars for a 48-note chromatic scale. The pattern causes chords, ascending and descending runs and melodies to be heard.

You may notice some interesting links between the visuals and the audio, especially if you are a musician. For example, when the pattern forms a 3-arm starfish, the chords you are hearing are diminished chords, which consist of minor thirds, an interval in which the notes are 3 chromatic steps apart. The chords you hear always bear this type of relationship to the pattern you are seeing, consisting of intervals which match the arrangement of arms.

I generated the audio using my sound synthesis program, Syd, which provides a very elegant way to accomplish this kind of thing. Unfortunately, Syd is horribly supported these days (I haven’t updated the Macintosh version since before OS X). Here’s a sample Syd patch which was used to generate the audio (I actually used 3 similar patches and mixed the results together).


Here’s are some other variations. The first few assign pitches based on the harmonic series, rather than the chromatic scale (thus the dot that represents the 10th harmonic actually sounds like the 10th harmonic). In these variations, the “chords” sound like single notes with a buzzier timbre, because when you combine harmonics, you are performing a kind of additive synthesis.

Whitney Music Box (harmonic version – fundamental = 55hz)

Whitney Music Box (harmonic version – fundamental = 20hz)

The above variation uses a lower fundamental which is close to the limit of human pitch perception.

Whitney Music Box (harmonic version – fundamental = 48hz, reversed)

The above variation reverses the assignment of the pitches, so that the lower fundamentals are heard more frequently.

Whitney Music Box (harmonic version – Palindrome)

The above variation uses both forward and reverse pitch mappings at the same time, to create a palindrome effect. The oscillators with the reverse pitch mapping are 180 degrees out of phase, which causes some nodes to cancel each other out, and creates some mildly interesting filtering effects.

Whitney Music Box (chromatic version – 88 tines)

The above variation uses 88 tines in a chromatic arrangement with alternating ascending and descending notes. Because there are more bumps on the wheel, there are more notes in the sequence.

Update 4/25/06:

I’ve generated a few more variations, which you can see here. I’ve begun using STK (the synthesis toolkit) instead of Syd to generate the audio, since this is more commonly available software that will allow other folks to experiment with this type of system. I generate the audio score using a Perl script which outputs note events in the SKINI format. I use the command-line utility calico to render the audio.

Update 4/26/06:

Nyquist, the Lisp-based music language, also provides an elegant way to generate this type of music. Here’s a sample nyquist score that I used to generate variation 11.

Update 5/2/06: this new post, I describe the nature of the star-fish patterns in more detail.

Update 9/22/11:

I added links to Whitney’s book in the blog post, and slightly modified the text for clarity. I changed the original Flash illustration to a GIF, so it can be viewed on IOS devices. I’ve updated the links to point to their current locations. You’ll find lots of sample code on Github.

62 Responses to “Visual Harmony”

  1. jbum Says:

    Great stuff Dan! I added a blog entry to highlight your work. Looking forward to seeing more!

  2. KrazyDad » Blog Archive » Dan Piker’s Whitney Variations Says:

    […] a recent comment here on KrazyDad, Daniel Piker shared some flash variations he made recently of the John […]

  3. ariyor Says:

    In the originals, as you move out from the centre each layer does one less rotation per cycle.

  4. Alex Sanchez Says:

    Hi, your work is very interesting. I am a music teacher doing research on some math/music ideas. Do you have a program like tone matrix ( where you can customize the scales utilized as well as the metrical grid? That would be very helpful!! Thanks so much!

  5. jbum Says:

    No, however you’ll find some scripts on the Whitney pages that can be modified to customize scales and so on.

  6. Sebastien Cazaudehore Says:

    Hello everyone,
    today I went on a weird quest. about 12 years ago (first problem) I accidently found a website/blog with real-time interactive music triggered by activating various parts of a fractal setup.
    of course, there was almost no words on this page which makes it impossible to track back. I know that htis was a long time ago, and chances are that the site does not existe anymore. but maybe someone heard about it and remember its name. any thoughts on the question and you can email me please.

  7. foxx314 Says:

    hi there,
    damn, i surely spend hours watching these harmonics =) very nice !!
    i’m interested in mathematics and noticed that the same patterns occur in the orbits of the mandelbrot set:

    click , in the applet choose “tools / mouse shows orbit”, and then click & drag your mouse near the border of the main cardiodid. exciting !! is there some similarity ??

  8. Sammy Says:

    Wow. That is absolutely brilliant. I’m kind of an aspiring music theory buff, and seeing the different patterns and numbers in this manner was really, REALLY awesome. It’s got a lot to say about resonance and harmonics. Beautiful work. Just plain beautiful.

  9. A very interesting experiment in sound & motion – “Visual Harmony” – The Whitney Music Box | The Beat-Play Experiment Says:

    […] Info from creator at […]

  10. SelflessGene Says:

    Hey, You’re on reddit!

    I’d love to see you put out a version where the dots moved based on the laws of gravity.

    Have : velocity = sqrt(m^2/((n+m)*r))

    where m is the mass of an arbitrarily massive invisible black hole at the center,

    n is the size of the planets (as estimated by the size of each circle)

    r is the distance between each planet and the center.

    Let me know if you plan on doing it!

  11. Squirrel Says:

    Wow! I love it! I found this form originally as pelotas.swf and got frustrated with not being able to control it…… so I made my own.

    I love your addition of the sound. I think I am going to sit here and let these play all afternoon.


  12. jbum Says:

    Thanks Squirrel!

    I’ve also done some Processing versions, which you’ll find here: