池之上響 Hibiki Ikenoue

Self-Playing Melodica

A machine that plays a melodica from MIDI

Self-Playing Melodica
Period
2023 – 2024

Overview

A machine that reads MIDI data and plays a melodica automatically. Instead of pressing the keys, it pulls the air valves with strings from the back of the instrument to sound each note. I built it as a school festival exhibit, and in the end it even performed together with a Tesla coil.

The machine playing Bad Apple!! (click to play; sound on)

How the idea came about

Left to itself, a robotics club exhibit ends up meaning something only to people who already know the field. Looking for an exhibit anyone could understand, one that would stop visitors in their tracks, I settled on music. A machine playing an instrument is interesting on sight, with no explanation needed.

Its predecessor was Music2023, an auto-performance program I had written in Python. You describe a piece in its own score format, and the software interprets it, synthesizes the waveform, and plays it through to the end. That is where I established the basics of processing musical data by machine, and from there I moved on to sounding a physical instrument.

The machine

System block diagramBlock diagram of the system
Close-up of the sounding mechanismClose-up of the sounding mechanism: one valve-pulling wire per key

The mechanism is a row of servos designed in Autodesk Inventor. Rather than pressing the keys from above, it pulls the air valves with strings from the back of the instrument. The keyboard on the front is completely unmodified, so a person can still play it as usual, even in a duet with the machine.

Circuit diagram of the systemCircuit diagram of the system (PCA9685 ×2 + ATmega328P)

The control board that drives the servos is also homemade, built around two PCA9685s and an ATmega328P. On the software side, the MIDI data is interpreted and converted into instructions for which key to press and when.

Editing MIDI dataThe MIDI sequencer I used and the converted MIDI data

The catch is that MIDI data is hardest to find for exactly the songs you want to play. So I also developed an image recognition system that reads an image of sheet music and converts it to MIDI. As long as you can get hold of the score, the machine can play any piece.

Analyzing a sheet music imageAnalyzing a sheet music image: notes are detected and converted to MIDI

Song selection was part of the exhibit, too. I chose widely recognized pieces such as Bad Apple!!, Senbonzakura, and Night of Nights. Precisely because the machine plays songs people already know, no technical explanation is required.

Duet with a Tesla coil

Testing the machineTesting with the air pump connected

In the end, it performed together with a Tesla coil that produces musical notes from its discharge sound. The piece reached the point of first surprising people with the technology and then entertaining them, which may make it the work closest to how I felt when I first started making things. As with my exhibit at the School Festival 2022, I designed it backwards from whether the audience would enjoy it.

Front view of the machineFront view of the machine, with 32 servos lined up behind the keyboard

← Back to projects