Audio Reactive LED Strips Are Diabolically Hard

In 2016, I bought an LED strip and decided to make it react to music in real time. I figured it would take a few weeks, but it ended up being a rabbit hole. Ten years later, the project has 2.8k GitHub stars, has been covered by Hackaday, and is one of the most popular LED strip visualizer projects available. People have built it into nightclubs, integrated it with Amazon Alexa, and used it as their first electronics project.

I'm still not satisfied with it.

Volume Is Easy

I started with non-addressable LED strips where I could control the brightness of the red, green, and blue color channels independently, but not the individual LED pixels. I tried the most obvious thing first: read the audio signal, measure the volume, and make the LEDs brighter when it's louder. These were all relatively straightforward time domain processing methods. Read a short chunk of audio around 10-50ms in duration, low pass filter it, map the intensity to brightness.

I assigned each color channel to a different time constant to get a kind of color effect. One color would respond rapidly to changes in volume, one would respond slowly, and one in the middle. You can get something like this working in an afternoon and it looks okay on an LED strip or a lamp with a single RGB LED.

It gets boring fast. All the interesting frequency information is lost and it works best on punchy electronic music. It is terrible on many other kinds of music where volume is not the most interesting feature. There is no understanding of what kind of sound the system is reacting to, just how loud it is.

I also had to implement adaptive gain control almost immediately. If you set a fixed volume threshold, the visualizer either saturates in a loud room or barely flickers in a quiet one. My favorite way to do this was with exponential smoothing a simple and effective filter that I used over and over in various parts of the code.

Although the time domain visualizer was okay, I found the limited output channels made the result unsatisfying. There is only so much information you can display on three color channels. Eventually, I switched to WS2812 addressable LEDs so that I'd have many more output features to work with.

The Naive FFT

The obvious next step was to use frequency domain methods. Collect a short chunk of audio, compute a Fourier transform (a mathematical tool that breaks audio into its individual frequencies), get frequency bins, and map them to LEDs. I had 144 pixels on a one meter strip, so I thought, 144 bins, one per LED. Then render the spectrum.

It kind of worked. I could tell right away that more of the audio was being captured compared to the volume method. But the result was deeply unsatisfying. Almost all of the energy was concentrated in a handful of LEDs, and most of the strip was dark.

I tried cropping the frequency range to use more of the strip. It helped a little, but I still felt that many of the LEDs were underutilized and that the FFT method was lopsided. I struggled with this for a long time.

Most people who attempt audio reactive LED strips end up somewhere around here, with a naive FFT method. It works well enough on a screen, where you have millions of pixels and can display a full spectrogram with plenty of room for detail. But on 144 LEDs, the limitations are brutal. On an LED strip, you can't afford to "waste" any pixels and the features you display need to be more perceptually meaningful.

Pixel Poverty

Pixel Poverty, Feature Famine, Compression Curse, whatever you want to call it, is the central lesson I learned and the reason LED strip visualization is so difficult. You might think that LED strips are simpler than screen-based visualizers, but the opposite is true. A screen-based visualizer has millions of pixels to work with, but an LED strip has hundreds at most. You can compute tons of audio features and display them all on the screen, and if most of them are uninteresting, it doesn't matter. As long as some of the features resonate with what a human perceives as interesting, the visualization works. On an LED strip, you have to be right about which features are worth displaying.

An LED strip is pixel-poor. A one meter strip might have 144 LEDs. That's it, and there's nowhere to hide. Nearly every single pixel has to be doing something that a human perceives as musically relevant. The margin for error is incredibly narrow.

This is what makes LED strip visualizers fundamentally harder than screen-based ones. I couldn't just display raw signal processing data. I had to understand how humans actually perceive music and build a perceptual model into the pipeline.

The Mel Scale

I started reading papers from the speech recognition field to understand how their signal processing pipelines worked. Speech recognition has spent decades figuring out how to extract features from audio that match human perception, because if you can't model what a human hears, you can't transcribe what they said, and that's where I found the mel scale.

Humans don't perceive pitch linearly. The perceptual distance between 200Hz and 400Hz feels much larger than the distance between 8000Hz and 8200Hz, even though both spans are 200Hz. Our brains are heavily tuned to the speech band between roughly 300Hz and 3000Hz, and much less interested in frequencies far outside that range.

The mel scale transforms frequencies from Hz into a perceptual space where pitches are equally distant to a human listener. Instead of mapping raw FFT bins to pixels, which spreads the perceptually important frequencies across only a few LEDs, I mapped mel-scaled bins to pixels.

The difference was night and day. The entire strip lit up. Every LED was doing something meaningful. That was the breakthrough. Everything else built on top of it.

What I realized is that the audio LED visualizer uses much of the same frontend as a traditional speech recognition pipeline. The mel filterbank, which speech systems use to extract perceptually relevant features before feeding them into a recognizer, is exactly what makes the LED strip come alive. I take the output of the mel filterbank and feed it directly into the three visualizations.

Smoothing, Flickering, and Convolutions

The mel scale solved the frequency mapping problem, but the raw output still flickered badly. Features changed too rapidly and the strip looked jittery and unpleasant. I needed the visualization to feel smooth and intentional, not noisy.

I applied exponential smoothing on a per-frequency-bin level, so each frame blends with the previous one. Features change gradually instead of jumping around. This eliminated the flicker without adding perceptible latency.

Then I discovered that convolutions (a mathematical operation that blends neighboring values together) were perfect for spatial smoothing. LED strips are 1D vectors, which makes them an ideal substrate for convolution operations. In university I learned the math of convolutions but the applications felt abstract. On the LED strip, it finally clicked. Different kernels gave me different effects: a narrow kernel for a max-like operation on adjacent pixels, wider kernels for gaussian blur. I could smooth the spectrum, soften transitions, and control how features blended spatially. I still think about convolutions in terms of LED strips today.

Both Sides of Perception

At this point I realized the visualizer needs perceptual models on both sides of the pipeline. On the input side, the mel scale models how humans perceive sound. On the output side, I needed to model how humans perceive light.

We don't perceive brightness linearly either. A raw linear mapping of audio energy to LED brightness looks wrong because our eyes have a logarithmic response. This led me into gamma correction (adjusting brightness values to match how our eyes actually perceive light) and color theory: RGB, HSV, LAB, sRGB, complementary colors. I learned that mapping frequency content to co