Less is more

I’ve come to appreciate that constraint drives creativity and innovation. The more options an artist has, the harder it could be to think about a solid idea. Without rules in music, the chances are so overwhelming that it might be near not possible to put in writing something listenable. It’s the principles and restrictions that give structure to an idea. In fact knowing when to interrupt those rules is just as vital as knowing what they’re.

Over the past three years I’ve been fascinated with using deep neural networks for emulating analog guitar amps and effects. I’ve written a handful of audio plugins and tested countless variations of neural networks for emulating guitar gear. Neural networks are CPU intensive, but with PCs you’ll be able to often throw more computer power at it to realize a more accurate sound.

For more information on how this works see my articles on neural networks for real-time audio.

But what should you desired to run a neural net on an inexpensive microcontroller, with a small (guitar pedal size) device, dedicated to simply running an audio effect and nothing else? Small project computers just like the Raspberry Pi are powerful enough to run an operating system, supporting applications, and a guitar effect, and have room to spare, which suggests wasted resources at higher cost. Most of these computers use microprocessors, versus microcontrollers. A micro-processor has external memory and I/O, and a micro-controller is a self contained computing system that may run programs on it’s own. Microcontrollers generally don’t have to be as powerful as a PC or smartphone, since it’s fully dedicated to doing one specific thing. Microcontrollers are perfect for a digital guitar pedal, but writing software for them is different from a smartphone or PC, and something I had never done before.

Enter, the Daisy Seed! The Daisy Seed is a $30 embedded platform for music, utilizing the Cortex-M7 ARM microcontroller. It has its own software library (libDaisy) and a DSP library for audio (DaisySP), all open source. For a guitar pedal, you wish an input audio buffer for connecting your analog guitar signal to the digital hardware, and an output buffer for matching impedance along with your amplifier. This ensures that your tone is preserved going through the effect. PedalPCB sells an interface board called Terrarium ($12) specifically for the Daisy Seed, and it even provides connections for as much as 6 knobs, 4 switches, 2 footswitches, input/output mono audio jacks, and 9 volt power (common to guitar effects).

At $42 for the audio hardware, I’m already well below the associated fee of the Raspberry Pi4 utilized in my previous NeuralPi guitar pedal. I purchased the boards, and because of my previous obsession with analog effects, had the remaining hardware readily available to construct the Terrarium pedal. Total cost was around $100 (with enclosure, electrical components, knobs/switches), and required basic soldering skills to place together. The existing projects other people had developed for guitar effects worked and sounded great on the finished Terrarium pedal.

BUT, my NeuralPi plugin can’t run on a microcontroller, so I needed to get creative. I made a decision to make use of the RTNeural engine to run the neural models and develop minimal c++ code around it using libDaisy and DaisySP. There are several codebases on the market for developing on the M-series microcontrollers, but I had no experience with these, so if in any respect possible I desired to follow what I already knew.

Now for those constraints I discussed: processing power, and memory. The M7 chip operates at a max of 480MHz, and at that speed can only utilize Flash memory, a tiny 128KB! (But to be fair, landing on the moon only took 74KB.) There are other memory areas you’ll be able to access on the Daisy Seed, but I wanted to maximise my processing speed to run the largest neural networks possible, to supply one of the best sound.

After I initially compiled my minimal c++ program with RTNeural, I blew past the 128KB limit. So then I reached out to the creator of RTNeural for help (as I even have persistently before!). He was able to cut back the compiled footprint by a big amount, allowing for many of space for the required code and model files.

Once my program would load onto the Daisy Seed, I used to be really curious what size model it could handle. I began with the NeuralPi models (LSTM size 20), nevertheless it was an excessive amount of. It caused the pedal to be unresponsive. Then I figured I’d start low and work my way as much as the limit, so I used a LSTM size 4, and success! It might run the effect (a capture of my TS-9 overdrive pedal) but I wasn’t glad with the accuracy. I decided that probably the most it could handle was an LSTM size 7, which is decent enough for many pedals, and might get some OK sounds on capturing lower gain amps. Have in mind that the Daisy Seed runs at 48kHz audio, which suggests 48,000 passes through the neural net each second!

This type of neural net sampling rate is impressive on any system, but I wanted it to sound even higher. Guitarists of recent years are accustomed to digital modelling that’s indiscernible from the true thing. Products like NeuralDSP’s Quad Cortex, IK Multimedia’s ToneX, and the more moderen Headrush Prime have given guitarists options to do what was previously only possible using a Kemper. Simply because my work is tailored to the DIY community doesn’t mean it must be any less capable! (Because the recent open source Neural Amp Modeler has proven.)

There may be one other type of Recurrent Neural Network (RNN) besides LSTM that I had not tested, the GRU (Gated Recurrent Unit). This network uses roughly 33% less processing power than LSTM for a similar internal size. I never did much testing with it because on a PC, the additional processing wasn’t as much of a priority.

The initial test with GRU was promising, I used to be getting even lower loss values on GRU’s than the identical size LSTM. Along with running faster, the GRU models used up less memory. I decided through trial and error that a GRU of size 10 was concerning the max I could run through RTNeural on the Daisy seed (vs. 8 with LSTM), and that difference improved loss values by half in some cases. I used to be getting loss values of lower than 0.01 for distortion pedals, and lower than 0.02 for low gain amplifiers.

With the present release, NeuralSeed (as I named it) can accurately capture most distortion/overdrive pedals (as much as 3 parameterized knobs) and low gain amps. Using no parameterized knobs stays probably the most accurate, but the flexibleness of controls for gain and tone is a cool option for my part. Techniques I previously learned have benefitted the general sound quality. These include improvements in my recording equipment, similar to using load boxes to record amps and re-amp boxes for pedals. Ranging from a pre-trained model (transfer learning) and using specialized input signals have helped improve accuracy within the model training. By utilizing a method called “distillation” (training a small model from a bigger model), I used to be in a position to capture the complex distortion of tube amplifiers at low to medium gain.

Here is the open source code on Github. When loaded on the Terrarium pedal, it has input/output level knobs, wet/dry mixer, 4 EQ boost switches, and as much as 3 knobs for controlling a parameterized neural model (gain, bass, and treble, for instance). The left footswitch either bypasses or engages the effect. The suitable footswitch cycles through the available neural amp/pedal models.

Developing for the Daisy Seed has made me appreciate the facility of recent day computers. The constraints of the M7 processor (which is a really capable microcontroller!) made me think outside the box to realize more using less. The NeuralSeed code is open source for anyone to check out on the Daisy Seed / Terrarium pedal, in addition to mod and make their very own tweaks and enhancements. There may be also a Colab script for training your personal models to run with the NeuralSeed software. Here’s a video demo of the finished Neural Seed pedal running several neural network models of actual guitar amps and pedals.

Special because of Jatin of ChowDSP for minimizing the RTNeural footprint and helping me solve the GRU implementation! Also because of the oldsters on the Daisy Discord for helping me understand the Daisy Seed and related software, and the Neural Amp Modeler Facebook group for providing training data.