software_synthesis

All posts in this category.

DMS – Physical Computing & Csound Final

For my Physical computing final project, I initially started by sketching ideas for things I am interested in making (more on this in this post). After some thought, I decided to mix two courses that really inspired me this semester, ‘Physical computing’ in ITP, and ‘Software Synthesis’ in Music Tech dept. of NYU. After brainstorming ideas (special thanks to Dror Ayalon & Roi Lev for that), I decided I would like to reimagine two concepts I am fairly interested in and touch both courses: Synthesis  and Modularity, and that’s how DMS – Different Modular System was born (at least conceptually).

Sketches of early stage ideas for the project
  • Conceptualisation:

I started by examing the features I think make modular synthesizers (and modular systems in general) powerful, both in terms of synthesis and interaction, alongside they’re downsides, and so the list began.

From http://gizmodo.com

Advantages:

  • Flexibility – modular systems are, by definition, a flexible instrument as they tend to allow more than a single configuration with the same components. This manifests into the audible realm too as you can process signals using the same components in different order and get different sonic outcomes (e.g filtering a signal before the delay vs. filtering after).
  • Interaction – modular systems tend to include more interaction from the user’s side, which can and at times does translate into a sense of customized ownership over the device. Simply put, rather than a synth, ‘it’s my special recipe using the synth’.

Disadvantages:

  • Terminology – This is a point that I’ll talk more about in regards to user play-testing, but it is worth mentioning at this point that the synthesizer arena in general and the modular one specifically, tend to overcomplicate terms to (at times) they’re mathematical and electrical origin, which sounds cryptic to most people and distances some users from even trying these systems.
  • Form Factor – As powerful as modular synthesizers are, they tend to be rather big installment devices. Thinking about these machines in a portable

*It is worth mentioning the list presents my perspective and does not imply that the current state of modular synthesizers is wrong or not valid, rather me trying to reimagine it in a different way.

Some of the references I used conceptually were:

  1. Little Bits Korg edition – a ‘bare boned’ Modular synth kit that is meant to serve as both an educational and musical modular system.
  2. Palette Gear – Modular controller eco-system for software and MIDI controller
  3. Korg MS-20 Mini – Semi modular synth ‘do it yourself kit’.
  4. Moog Werkstatt – Another semi modular synth ‘do it yourself kit’
  • Design:

This project imposed some big fabrication challenges for me, and due to that I decided to start the ‘making’ process with design and fabrication of the units.

Magnets, magnets and magnets!

Upon play-testing and discussing the project with both Danny Rozin and Ben Light, I decided I would start with building 2 modules that work, the main hub (i.e ‘the Brain’), and a second effect unit (i.e ‘the Mouth’). On the first Iteration I had around 6 magnets on each of the boxes, and after a Eureka moment decided to have an Arduino in each box, and let them communicate over serial. The initial bill of material looked something like the following:

I started the process by cutting acrylic top panels and leaving the exact diameter of the magnet that would later on be glued to the top. I used the protective plastic rings that came with the magnets (i.e separators), to elevate the magnets to 1mm above the plastic panel in order for them to always connect without disturbance (thanks Ben Light).

Final drilling of the cables soldered to a spring that pushes against the magnets

And finally, I soldered the wires onto springs that push against the glued magnets and pass the electrical energy. One note on this, is that as Ben Light mentioned to me, soldering directly to the magnet tempers with the magnet’s electrical properties and therefore might introduces unknown interferences to the electrical signal, the spring method however works fairly well, just remember to use epoxy.

As I mentioned before, terminology is something I really wanted to simplify into icons and language people feel comfortable interacting with (or at the very least doesn’t scare them away), and so for the synth’s interface I had multiple iterations, each time changing the icon or the text, and letting people react to that, this is how the final interface looks before the actual cutting and etching process

Last but not least, here are the two units after the finished fabrication process

Physical computing process:

The main reason for choosing the Arduino Mega for the brain module was it’s ability to both communicate using serial communication to the computer (in my case a Csound application), and use the additional serial ports (16/17), to communicate to another smaller Arduino in the mouth module.

The code below demonstrates the communication function on the brain module for sending the events to the computer. I use single byte messaging and divide 0-255 to all the synth parameters

A feature that really helped me and is both related to fabrication and implementation was putting a serial switch in the 2nd module that way I can change between Module-to-Module communication and sending new code to the 2nd module without disconnecting the serial lines.

  • Building the synth:

As this is my final project for both Physical Computing and Software Synthesis I used Csound, a synthesis library and engine to build all the actual sound generating logic.

I started by laying down the communication logic, the serial port, using Csound’s serialBegin and serialRead opcodes (the Csound name for logic functions built in the library).

After laying down the serial communication I started building the oscillators using Csound’s poscil opcode while storing the actual wave tables values for the waveshapes in an ftable.

During the making of the actual synth I discovered some functionality that I was really missing, for example the Arduino’s map(); function didn’t have any sibling on the Csound side, and so I challenged myself to extend Csound and build two mapping functions I used to map the Arduino serial coming in to different parameters in the Csound instrument I created.

 

After a lot of explanations, here is an image and a demo composition using the final synth

This is my final composition made entirely with the synth using Ableton Live as a looper and a MIDI controller hooked up to the synth.

  • Links:

Link to the Arduino Mega and Uno code

Link to the Csound program

  • Moving forward – things I would like to improve next:
  1. I would like to try and implement the synth on an embedded system that way I can have everything running from the box itself.
  2. I would like to build more modules to demonstrate more aspects of the synthesis process (e.g ‘the Spine’ as an arpeggiator).
  3. Implementing a better method to detect when the modules are connected and disconnected from the brain

*Special thanks Tom Igoe and Jean-Luc Cohen, this process has both taught me a lot, and challenged more throughout. Your help and guidance made this possible!

Physical Computing – Midterm

ezgif-4057205676

For our midterm project, I was paired with Amanda MJ Lee. We sat down and started sketching out ideas, which after a while, seemed started looking very obvious, most of our ideas dealt in some way or another with sound & music, and so we decided to make a musical instrument, luma a color-reactive audio synthesizer that uses color plates (or discs) to create musical patterns in real-time.

Sketch
Sketches of different ideas after brainstorming
  • Conceptualisation:

One of the points that kept coming up while discussing ideas, was the ability to use a well-known music consumption tool (e.g Turntable, Gramophone or Phonograph) and repurpose its interaction to create a musical instrument. This discussion turned into our project, which used the Turntable as means of symbolic relation to music, but repurposed its interaction to serve as a musical creation tool.

We decided to build upon this idea, but instead of having the device motorized, divide the instrument into two decks, rhythmical and melodic, in which only the rhythmical is motorized requiring the participant (or better yet musician) to learn and develop his own way of playing it.

ezgif-565323512

  • Design:

With the aforementioned in mind, we started thinking about visual references, we wanted the instrument to feel timeless (classic), yet have a very organic feel, since the color plates will introduce playfulness to the visual experience of using the instrument. Brown was a good visual reference

sk6_braun_450pxls
Visual reference – Bruan personal turntable

 With that in mind, we started assembling a ‘look’ and decided we are going to use a wooden box and acrylic beige cover, to convey this ‘classic look‘.

  • From sketches to fabrication:

What we used (i.e our bill of materials):

  1. Container Store Drawer Organizer Bamboo (6″ x 15″ x 2″ h) –  7.99$ – link
  2. Arduino Mega Rev 3.  – $45.95 – link
  3. 2x Color Sensors – 9.99$ each – link
  4. Linear Potentiometer (B50K) – 2.99$ – link
  5. 60mm Slider (Phidegts 1112) – 11$ – link
  6. 5 Sheets of acrylic in different colors – 60$
  7. Some basic breadboard wires

After we got all the parts we started thinking about how we would assemble our enclosure. During this discussion we realized, there were a couple of challenges everybody around us seem to be facing too, and so Amenda suggested it would be cool to try and tackle these as well for the benefit of everyone, and so to the 3D printer we GO!

img-20161024-wa0005
Measuring all the components and prototyping design ideas with cardboard

We 3D modeled all the mounts inside the case that hold the components, but we also decided to use the 3D printer to tackle design issues. For instance, as we needed to drill a USB hole inside the wooden box to connect the Arduino, we realized drilling a square hole was a mission impossible for us, and so decided to design a circular USB type B adapter that would fit inside the hole the drill press created  (the adapter can be found in Dror Ayalon and Mint’s awesome project Video Manipulations too, YAY we helped ITP)

* All 3D models used in the project are available here

We used a hot glue gun to glue the 3D printed mounts into the enclosure and started placing the sensors and components using double sided (really sticky) tape. Following that point, we started designing the interface, controls and laser cutting the actual discs – one lesson learned from that process is that when you prepare to the smallest detail, it actually is a very enjoyable one.

output_4psna4

  • Coding & Implementation:

We chose to have the Arduino analyze all the inputs from the sensors, potentiometers and sliders and communicate to the computer over the serial port. With that in mind we started placing all the logic on the Arduino side first, and later on moved to creating the synthesizer.

breadboard
Arduino breadboard illustration

We started coding with only one deck assembled, as we were multithreading design and code, trying to touch up and implement together. The fact we had only one deck available at the start, actually gave birth to a cleaner coding approach where we decided to break the functionality into small functions that deal with every part of the functionality chain separately. Here is our final loop function with comments to enable control every step of the process and even scale it very fast (1 deck or two deck, is only commenting and uncommenting a function).

Another thing we implemented at this point was the ability to tell whether a color has just started. We did this with the same logic of a button change press just utilizing a color range as the changing into/out of range which is handled by the matchLastColorState();

A couple of things we realized on the way were:

  1. If you code individual blocks of logic, it is easier to debug them separately
  2. If you communicate over serial with binary data, it is useful to have a function that you can switch on and off to debug with strings, that way you can actually read it
  3. When using sensors that are affected by ambient factors (light, sound…etc) prepare to test extensively (and then test more).
  • Building the synthesizer:

As I am also taking a software synthesis course this semester in Steinhardt’s Music Technology department, I suggested we use Csound, the system used in the course, Amanda was in and so we started writing the synth in Csound. Some of the challenges we had to face was the binary serial communication between the Arduino and Csound, building interesting instruments that would play, and deciding on the logic at which we trigger different notes, so it doesn’t repeat the same note whenever a color is detected.

ezgif-955914941

We ended up building a synth that uses 10 oscillators and 5 envelopes to create rich ambient and percussive sound textures, iterating over a pentatonic scale which makes the playing experience more engaging.

The code for the Arduino app can be found here, and the code for the synthesizer can be found here

Here is the final graphic layout we made to explain the project:

illustration2-01

Demo video:

And here is the live demo we did in class on presentation day: