Some of the most interesting technical innovations in this field can
be seen in the sensory devices used to gather information from the real
world. The purpose of these devices is to measure the actions of a person,
often a performer, and to make the data available to a device that can respond
intelligently to what is happening.
In terms of commercially available devices, we have attempted to list
the best known and most readily available sensors here. For an exhaustive
list, see Axel Mulder's excellent Human
Movement Tracking Technology paper and its addendum.
This page is divided into three sections:
Making Your Own Sensors
There are two reasons that you may need to create your own sensory devices.
The first strike against commercial sensors is probably their expense. We
are artists after all, and that means we're usually short on cash. The second
reason is that off-the-shelf objects may not measure the specific actions
that you are looking for, or may not measure them in the way that you need.
This section is designed to help you find the information you need to create
simple and relatively inexpensive interactive sensory systems.
If you are a beginner, we strongly encourage you to start at Dan O'Sullivan's
Computing pages. This "hands on guide for artists" goes into
great detail, from choosing sensors to soldering to interfacing them to
Next, we would like to list a few very simple ideas for beating off-the-shelf
components into submission for your own purposes.
Drum Controllers and Drum Machines
Many drum controllers (Roland Pad-80) and some drum machines (Alesis
HR-16) have inputs for external triggers. These inputs can be connected
to piezo sensors, which respond to vibration or impact. You can purchase
these sensors at Radio Shack very inexpensively, and then hook these up
to the inputs on the drum controller or drum machine. Once you have done
so, striking the piezo should generate midi note on / off messages. Because
they are small and thin, the sensors can be attached to a number of surfaces
(walls and floors), embedded in costumes, etc.
MIDI Fader Boxes
There are a number of commercially available MIDI Fader Boxes, which
usually sport a number of knobs or sliders. Moving the slider causes some
kind of MIDI information (often a continuous controller message) to be sent
out of the box. It is fairly easy to crack open these boxes, disconnect
the existing faders and replace them with your own. This requires a bit
of courage because it is possible to destroy the object in question if you
are unskilled with a soldering iron, etc. The procedure is outlined below.
There are a number of possible applications, but obvious choices include
measuring the angle of joints, how a viewer is touching parts of an installation,
Note that we do not assume any responsibility for damage that occurs
when modifying devices in the way described below. If you are at all unsure
how to do the following procedure, either don't do it or find a knowledgeable
friend to help you!
- Disconnect power from the device and open its enclosure.
- Look for the wires that are connected to the sliders or knobs (we'll
call them faders from here on.) There should be three wires connected to
- Make a diagram of the wires connected to the fader. Better yet, number
the contacts with a permanent marker and then, using a piece of tape, mark
each wire with its matching number.
- Unsolder the wires from the fader.
- You will need to find out the resistance value of the fader. If there
is a rating printed on the fader itself, i.e., 100K or 10K or something
like that, you may not need to do the following procedure. If not, you
can measure the resistance yourself using a volt-ohm-meter.
- To find out the fader's resistance value, measure the resistance between
all three pairs of contacts. Two pairs will change resistance when you
move the fader. One pair will not. The resistance of the fader between
these two points is the fader's rated resistance. At this moment, the connection
that you are not touching is called the "wiper." Make note of
- At your local electronics store or Radio Shack, purchase a potentiometer
with the same rating as you found in the previous steps. Either a rotary
or linear version is OK whatever suits your purpose.
- You will want to solder your potentiometer in place of the fader
that you disconnected. On the potentiometer that you purchase, you can
determine the "wiper" in exactly the same way that you did above.
On rotary potentiometers, the wiper is always the connection in the middle
of the group of three.
- Attach the wire that used to go to the wiper on the device's fader
to the wiper of your new potentiometer. Then attach the other two wires
to the remaining two connection points it does not matter which one.
That's it. You should be able to turn on power to your device, and use
your new fader to generate MIDI in the same way in which the old fader did.
Many artists have created their own sensory systems to suit their specific
needs. These creations range from the simplest hybrids of multiple off-the-shelf
items to systems that were created from the ground up. We have listed several
of the devices that we know of here, in the hopes that they may serve as
source of ideas and inspiration.
If there is an significant, non-commercially available system that we
have omitted here, please let us know.
MidiDancer, created by Mark Coniglio, allows the movements of a performers
body to generate MIDI information. The sensors measures the flexion of
several joints (wrists, elbows, hips, and knees) of the body. The resulting
movement information, measured is sent via a wireless link to a box that
decodes the information and passes on to a computer in the form of MIDI
continuous controller information.
- BodySynth. Developed by Chris van Raalte, the BodySynth uses EMG (Electro-Myographic,
i.e., muscle contraction) sensors to generate MIDI information. The resulting
information is sent via a wireless link to a decoder, which then passes
the information on to a computer for processing. This unit was commercially
available at one time, though I am not sure if it is any longer in production.
Please send an email if you have
more current information.
Commercial Devices (In Production)
- IceCube. Axel
Mulder's IceCube is a input device that can measure a wide variety of input
sources. Sensors available for the IceCube include: illumination, , pressure
(touch), temperature, short range (up to 7 cm) proximity, rotation and
with several others in development.Output is in the form of MIDI System
Exclusive messages. The package also includes MAX objects designed specifically
to take advantage features of the IceCube system.
- Lightning. Don Buchla's lighting
allows the computer to track the position of two "wands" in space.
The wands (actually small infra-red transmitters) are generally held in
the hands of the performer, and moved about in front of a receiver. Thunder
can use the position of the wands to generate various kinds of MIDI data,
including note on/off and continuous controller information.
- SensorLab. Created
at STEIM in Amsterdam, the STEIM
SensorLab is a small, general purpose, analog to MIDI interface for the
prototyping of musical instruments and interactive control systems. This
box has 32 channels of analog to digital conversion, 2 ultrasound inputs
for measuring distance between sensors, 128 switch inputs and more.
- Very Nervous
System. David Rokeby's
system has received a lot of attention and has been used in several notable
installations. VNS is a non-invasive motion tracking system that analyzes
input coming from a video camera. It can sense motion in a space and where
that motion occurs. Output from the VNS via a SCSI connection, meaning
that you connect it to your computer like a SCSI hard disk drive or scanner.
Objects are provided that allow access from the MAX programming environment.
- Theremin. Almost
certainly the first electronic instrument, the Theremin was created in
1919 by its namesake Leon Theremin. The device uses two metal antennae
to sense the motion of the hands in space near the device. One antenna
controls the volume of the device, while the other controls the pitch.
Bog Moog's company Big Briar now
makes a Theremin that outputs MIDI information. There is much information
to be found on the net, starting with Jason Barile's Thermin
Commercial Devices (No Longer In Production)
Because of the small number of creators making use of alternative controllers,
many have come and gone. Still, you may find one of these items wandering
about looking for a home so we felt it was important to mention some of
these devices here.
- Airdrums. Created by inventor Pat Downs, the Airdrums were a pair of
MIDI batons, about the size of claves, that send messages when moved on
their axes in various directions. Each axes was sensitive to motion in
six axes, and each axis able to send a pre-programmed series note on events
and other MIDI data. (You can find a review in Keyboard Magazine's January
- Gold-Brick Interface. This box converted signals from a Nintendo Power
Glove (see below) into MIDI. A fair number of people seemed to be using
this system at one time, so you might be able to find one used.
- PowerGlove. The Mattel PowerGlove was originally marketed as an interface
to Nintendo video game systems. But its ability to track both position
of the hand in space (using ultrasonic sensors) and the bending of the
fingers led to almost instant hacking by many artists. In its off-the-shelf
configuration, the glove required custom hardware to get at the data. Later
the Gold-Brick Interface (see above) converted the output into MIDI, which
made it much easier to use. One important development was the Chris Hand
has written a good summary
of the device and its history, and a PowerGlove FAQ
has been compiled by J. Eric Townsend.
Sensors | Interactive
Languages | Presentation Devices | Roll Your Own | DT&Z
Maintained by Scott Sutherland <firstname.lastname@example.org>