A few have asked for more details about my hybrid chess/music system, so here we go.

To recap a previous post, I’m working on a performance piece for the June 26 Cranks, Cams and Computers concert in Eagle Rock. In this piece, two performers will reenact a famous game of chess (Kasparov vs. Deep Blue, 1997, Game 6). As they play, a computer program I’ve written monitors each move, and tries to figure out the best possible move to make in response. During this analysis, it produces music on the fly. The music is a fairly literal translation of what the computer is thinking. The music for each move lasts about 10 seconds, but represents only a fraction of a second of the thinking the computer was doing. When the next move is made on the board by a performer, the music from the previous move’s thinking is interrupted, so we can hear the thinking for the following move.

The chessboard the players are using is a Novag Citrine, which is a wooden chess board with internal electronics and magnetic pieces. The board has a PC serial interface, which is connected to my computer via a serial-to-USB adapter.

My program converts chess information to music by producing MIDI notes. Each MIDI note has 4 elements that I can control, based on various aspects of the chess analysis: pitch, loudness, timbre, and duration. The triggering of the notes, and those parameters is based on the computer’s chess search.

CHESS SEARCH

As the computer evaluates which move to make, it searches a tree in memory, using something called the minimax algorithm. This tree starts with all the possible moves the computer might play. Each of those moves leads to a node (a branching point), from which branches all the possible moves the opponent might play in response. Each of those moves leads to a new node from which branches all the possible moves the computer might make in response, and so on. After 5 or 6 branchings (or look-aheads) the computer evaluates each of the moves (5 moves ahead or so) based on various factors, the most significant being how much material has been lost or gained. As it backtracks through the tree, it chooses a move for each branching point, assuming that the opponent will make the worst possible move (or the best move for the opponent) — minimizing the score and that the computer will make the best possible move — maximizing the score. Hence the name “minimax”. This would be a lot clearer with an illustration: you’ll find one here.

As the computer searches this minimax tree, it produces MIDI notes, which correspond to the moves that are being evaluated. Since the tree has a branching factor of about 30 (there are typically about 30 chess moves each player can choose from), only the first 3 or 4 levels of the tree are converted to notes (this produces about 30^4 or about 800,000 potential notes per move). For the deeper levels of the tree, which produce potentially enormous numbers of notes, only moves which are considered interesting (which produce high or low scores) are heard as notes. Since those interesting moves are often the same, regardless of the moves that led to them, (so called “killer moves”), these deeper levels often take the form of an ostinato (repeating note) pattern. When it isn’t ostinato, the chess board is more unstable and a variety of moves lead to very different outcomes.

There are two different triggering systems used, depending on which side is playing, white or black. For the white side, I play the tree nodes in the order in which they are encountered (a depth-first search). For the black side, I play the different levels of the tree simultaneously (closer to a breadth-first search) which produces a more contrapuntal style, with each tree level having a different simultaneous melodic line.

PITCH

The pitches of the notes, are based on the the chess moves that are being considered as the computer searches the tree. Each chess move consists of a source and a destination square, such as E2-E4. Rather than actually playing an “E” to represent the square E2, I’ve assigned unique ascending chromatic pitches to each of the squares on the chessboard, with A1,B1,C1… being the lowest notes, and …F8,G8,H8 being the highest notes. Since there are 8×8 or 64 squares on the chessboard, this gives us a range of 64 notes, or a little over 5 octaves.

The notes are transposed a little higher when the computer thinks white is winning, and a little lower when it thinks black is winning. They are also transposed an octave or two up or down, to fit within the range of the instrument that is playing them, as necessary.

When the players make their moves on the chessboard, those moves are played loudly with both the source and destination squares heard, as a kind of announcement.

Then, as the tree is being searched, only the destination square are used for most moves, since there are thousands of notes being produced.

TIMBRE

The timbres of the notes are controlled by selecting different MIDI channels on my sampler, which, for this piece, holds 15 different instruments. There are three instruments which are used by both opponents (bass, piano and congas) and the remaining 12 instruments are split between the two opponents, with white (Deep Blue) getting more percussive sounds, and black (Kasparov) getting more string and wind instruments.

As each move is made, 3 or 4 instruments are chosen randomly from the ones available to that player.

A different instrument is used to represent each level of the search tree.

LOUDNESS and DURATION

High levels of the search tree (closer to the root) get louder and longer notes. As we go deeper into the search tree, the notes get faster and softer, so that we can play more of them as they multiply.

Here’s an excerpt of the piece that I recorded a few weeks ago. At this time, some of the logic I described above was not yet in place.

After the Eagle Rock concert, I’ll post a more recent recording of the piece.

This entry was posted on Sunday, June 14th, 2009 at 4:27 pm and is filed under Linkydinks. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.