By Michael Gamble, Partner & Ecosystem Lead, Arm
As a software engineer and music producer, I’m at all times exploring how technology can expand creative expression. That curiosity recently led me to construct a private sound generation app that runs directly on-device—powered by an Arm-based CPU and open-source generative AI models. It’s fast, private, and enables me to generate studio-ready sounds from an easy prompt, all inside seconds.
This project brings together one of the best of several worlds:
- The Stable Audio Open model from Stability AI, sourced from Hugging Face
- Execution powered by PyTorch and TorchAudio
- A quick, efficient pipeline that runs natively on Arm-based CPUs
- A seamless creative handoff to Ableton Live
A Latest Type of Creative Companion
After I’m deep in a music project using Ableton Live, I don’t need to interrupt my workflow to dig through libraries or browse sound packs. I wanted a tool that might meet me where I’m—right within the flow.
Now, I can simply describe the sound I’m imagining (“analog bassline,” “cinematic riser,” “lofi snare”), and inside seconds, the generated .wav file appears in my Ableton browser. From there, I can tweak it, loop it, or turn it into an instrument.
Every sound is exclusive. Nobody else will generate exactly what I do. That sense of non-public ownership fuels my creativity.
Powered by Arm: On-Device, On-Demand
This sound generator runs entirely on-device using Arm-based CPU technology—no GPU, no cloud inference, no latency. Because of Arm’s efficiency and performance-per-watt, the app stays responsive even during multi-step diffusion runs.
The generation engine is built on:
Sample Code: Optimized CPU Generation
To maximise performance on Arm CPUs, I enabled full thread utilization:
torch.set_num_threads(os.cpu_count())
To keep up low memory usage across generations:
if gen_count % 3 == 0:
gc.collect()
print(f"Memory cleared at generation {gen_count}")
Core generation loop, tuned for speed and efficiency:
output = generate_diffusion_cond(
model,
steps=7,
cfg_scale=1,
conditioning=conditioning,
sample_size=sample_size,
sigma_min=0.3,
sigma_max=500,
sampler_type="dpmpp-3m-sde",
device=device
)
Device Flexibility: CPU, Metal, CUDA
Although optimized for CPU, this system also can run on Metal (Apple Silicon) or CUDA if needed:
device = "mps"
model = model.to(device).to(torch.float32)
Seamless Workflow with Ableton Live
The tool outputs .wav files on to a project folder monitored by Ableton Live. Here’s a sample CLI interaction:
Enter a prompt for generating audio:
Ambient texture
Enter a tempo for the audio:
100
Generated audio saved to: Ambient texture.wav
I immediately see the file show up in my browser inside Live, able to be arranged, modulated, and transformed.
Why This Matters
This project is a private prototype—but it surely’s also a window into the longer term of content creation. With efficient, on-device AI inference on Arm CPUs, artists and developers can:
- Stay in creative flow without waiting on cloud resources
- Ensure data privacy and full ownership of outputs
- Extend AI tools into edge devices, DAWs, and latest creative interfaces
That is what happens when open-source innovation meets efficient compute: real-time generative power, accessible to each creator.
Explore the ecosystem that made this possible:
