Haunted Garden has come a long way since its humble beginnings in a high rise condo in Florida. It originally started as an Aha! moment. I was wondering what it would be like to add simple tones to harmonize a sonic environment when I thought: "I know how to do that!"

At the time I was walking on the beach while visiting my parents in Florida for thanksgiving. By thanksgiving dinner I had it running. It was very simple: detect notes in the incoming sound stream, find their harmonic series, and choose three notes at random from the series, then play them as sine waves. It took awhile to get the levels right. It turns out that a high frequency sine wave is rather annoying when played looudly. The volume needs to attenuate proportionally to the frequency.

I got the idea to control the color of lights according to the sounds long before I actually had any lights to control. I first wrote the software to change the color of rectangles on the screen. The idea involved a key estimator. I decided to figure out the musical key of the sounds it was hearing and use those to map to the lights.

Then a friend of mine invited my to a guerilla light art show in a park on Lake Merrit in Oakland. I brought a laptop with Haunted Garden changing the screen colors based on the sounds it was hearing. There I met Jeremy Lutes who had some color-addressable LED bars he had made, and we decided to connect them to the installation.

In 2005, I received a Borg2 grant to bring Haunted Garden to Burning Man. I decided to use the icosahedral sound system I had developed for Heavens Speak. I used the grant money to buy a mac mini to run the system on, put together a new 12 channel amp using car stereo amps.

The installation was presented with no user interface and no visible computers. Participants merely happened upon a tent out at the edge of the city. There was a foam floor, large stuffed animals to lounge upon, and soft ambient music playing. Many people thought of it as just a chill room oasis out of the harsh elements. If they cared to be quiet & listen to the music, it shut up. When they spoke again, it started playing. Eventually word got around, and by the end of the week people were coming specifically to interact with it, occasionally bringing instruments.

Back from the playa, I decided to teach Haunted Garden to play piano. The concept was simple: to generate sine waves, I required the frequency, volume, and duration. The math to derive a midi note number from the frequency is quite simple: a scaled log with offsets. Then I could send the midi note commands to any synthesizer or sampler. In this case I used Reason to design and play a number of software instruments, and then have it make piano, cello, and synth pad music to fit a room.

Playing instruments changed Haunted Garden in an interesting way. Midi uses an equal-tempered scale -- same as a piano, while the sine-wave harmonics calculated from detected notes used a more natural temperment -- integer ratios of their frequencies. When switched from one to the other, the difference was readily apparent.

Now that there was a possibility of playing multiple instruments, I decided to give that choice to participants. So I got a touch screen and wrote a simple interface to give users control over which instruments were played for which notes.

The interface went through a number of iterations, and was presented at the Double Vision show at the Culture Catch Salon and at Maker Faire.

The next step was to add field recordings into the mix. There's no reason to limit Haunted Garden to only responding to the present moment, so I put in a set of recordings I had done in Tibet, Boulder, the 10 bus in San Francisco, etc, and wrote a little interface to allow users to choose a field recording to play and decided whether Haunted Garden responds to the current environment, the field recording, or somewhere in between.

For the most recent show, at ON Gallery in April 2009, I added a new feature I call Instrument Resonance. The idea is that Haunted Garden generates notes based on the sounds it hears, then there are a number of virtual instruments. Each one knows the notes it has played, and is asked how well the next note would fit that melody. The note is given to the instrument with the best match.