ITP Blog

My journey in making things

Physical Computing – Midterm


For our midterm project, I was paired with Amanda MJ Lee. We sat down and started sketching out ideas, which after a while, seemed started looking very obvious, most of our ideas dealt in some way or another with sound & music, and so we decided to make a musical instrument, luma a color-reactive audio synthesizer that uses color plates (or discs) to create musical patterns in real-time.

Sketches of different ideas after brainstorming
  • Conceptualisation:

One of the points that kept coming up while discussing ideas, was the ability to use a well-known music consumption tool (e.g Turntable, Gramophone or Phonograph) and repurpose its interaction to create a musical instrument. This discussion turned into our project, which used the Turntable as means of symbolic relation to music, but repurposed its interaction to serve as a musical creation tool.

We decided to build upon this idea, but instead of having the device motorized, divide the instrument into two decks, rhythmical and melodic, in which only the rhythmical is motorized requiring the participant (or better yet musician) to learn and develop his own way of playing it.


  • Design:

With the aforementioned in mind, we started thinking about visual references, we wanted the instrument to feel timeless (classic), yet have a very organic feel, since the color plates will introduce playfulness to the visual experience of using the instrument. Brown was a good visual reference

Visual reference – Bruan personal turntable

 With that in mind, we started assembling a ‘look’ and decided we are going to use a wooden box and acrylic beige cover, to convey this ‘classic look‘.

  • From sketches to fabrication:

What we used (i.e our bill of materials):

  1. Container Store Drawer Organizer Bamboo (6″ x 15″ x 2″ h) –  7.99$ – link
  2. Arduino Mega Rev 3.  – $45.95 – link
  3. 2x Color Sensors – 9.99$ each – link
  4. Linear Potentiometer (B50K) – 2.99$ – link
  5. 60mm Slider (Phidegts 1112) – 11$ – link
  6. 5 Sheets of acrylic in different colors – 60$
  7. Some basic breadboard wires

After we got all the parts we started thinking about how we would assemble our enclosure. During this discussion we realized, there were a couple of challenges everybody around us seem to be facing too, and so Amenda suggested it would be cool to try and tackle these as well for the benefit of everyone, and so to the 3D printer we GO!

Measuring all the components and prototyping design ideas with cardboard

We 3D modeled all the mounts inside the case that hold the components, but we also decided to use the 3D printer to tackle design issues. For instance, as we needed to drill a USB hole inside the wooden box to connect the Arduino, we realized drilling a square hole was a mission impossible for us, and so decided to design a circular USB type B adapter that would fit inside the hole the drill press created  (the adapter can be found in Dror Ayalon and Mint’s awesome project Video Manipulations too, YAY we helped ITP)

* All 3D models used in the project are available here

We used a hot glue gun to glue the 3D printed mounts into the enclosure and started placing the sensors and components using double sided (really sticky) tape. Following that point, we started designing the interface, controls and laser cutting the actual discs – one lesson learned from that process is that when you prepare to the smallest detail, it actually is a very enjoyable one.


  • Coding & Implementation:

We chose to have the Arduino analyze all the inputs from the sensors, potentiometers and sliders and communicate to the computer over the serial port. With that in mind we started placing all the logic on the Arduino side first, and later on moved to creating the synthesizer.

Arduino breadboard illustration

We started coding with only one deck assembled, as we were multithreading design and code, trying to touch up and implement together. The fact we had only one deck available at the start, actually gave birth to a cleaner coding approach where we decided to break the functionality into small functions that deal with every part of the functionality chain separately. Here is our final loop function with comments to enable control every step of the process and even scale it very fast (1 deck or two deck, is only commenting and uncommenting a function).

Another thing we implemented at this point was the ability to tell whether a color has just started. We did this with the same logic of a button change press just utilizing a color range as the changing into/out of range which is handled by the matchLastColorState();

A couple of things we realized on the way were:

  1. If you code individual blocks of logic, it is easier to debug them separately
  2. If you communicate over serial with binary data, it is useful to have a function that you can switch on and off to debug with strings, that way you can actually read it
  3. When using sensors that are affected by ambient factors (light, sound…etc) prepare to test extensively (and then test more).
  • Building the synthesizer:

As I am also taking a software synthesis course this semester in Steinhardt’s Music Technology department, I suggested we use Csound, the system used in the course, Amanda was in and so we started writing the synth in Csound. Some of the challenges we had to face was the binary serial communication between the Arduino and Csound, building interesting instruments that would play, and deciding on the logic at which we trigger different notes, so it doesn’t repeat the same note whenever a color is detected.


We ended up building a synth that uses 10 oscillators and 5 envelopes to create rich ambient and percussive sound textures, iterating over a pentatonic scale which makes the playing experience more engaging.

The code for the Arduino app can be found here, and the code for the synthesizer can be found here

Here is the final graphic layout we made to explain the project:


Demo video:

And here is the live demo we did in class on presentation day:

Hi I am Or, I am a director, developer and artist. My current interest in research is sound interaction, computer vision & immersive media development.

Leave a Reply

Your email address will not be published. Required fields are marked *