What a Music Visualizer Actually Shows You

Everyone who used a computer in the early 2000s has a memory of it: a track playing, a window full of pulsing colour, and ten minutes lost staring at it. Winamp shipped with two of them. Windows Media Player and iTunes had their own. MilkDrop became a genre in its own right. Visualizers have never really gone away, they just stopped being the thing everyone talked about. Which is a shame, because behind the swirling shapes is a genuinely interesting question: what is the picture of?

Two ways to look at sound

Digital audio is a long list of numbers: tens of thousands per second, each one the position of the speaker cone at that instant. There are only two fundamental ways to turn that list into a picture, and every visualizer ever built is a variation on one or both.

The first is the waveform: draw the numbers as they arrive, left to right, amplitude up and down. It's what an oscilloscope shows. Loud passages are tall, quiet ones are short, a kick drum is a sharp spike followed by a wobble. It shows you when things happen and how loud, but tells you almost nothing about what, because a bass note and a cymbal can look like the same squiggle at a glance.

The second is the spectrum: how much energy is present at each frequency right now. This is the row of bouncing bars, and it's far more informative, because it separates bass from treble, instrument from instrument. Getting from a list of samples to a spectrum takes some maths, and that maths has quirks that explain almost everything odd about the way visualizers behave.

The FFT, briefly

The tool is the Fast Fourier Transform. Take a short window of samples, say 2048 of them (about 46 milliseconds at CD sample rate), and the FFT tells you how much of each frequency was in that slice. It's the same idea as splitting white light into a rainbow with a prism, and a browser's audio engine does it dozens of times a second.

Two consequences follow, and they're the source of most of the visualizer's personality.

The bins are linear, but hearing is logarithmic. A 2048-sample FFT divides the spectrum into 1024 bins spaced about 21.5 Hz apart. That sounds precise until you notice how it lines up with music. The entire bass region, 20 to 200 Hz, gets about eight bins. The top octave, 10 to 20 kHz, gets four hundred and sixty-five. Draw the raw bins as bars and the picture is almost all treble, with the whole musical foundation crammed into a sliver on the left. Every decent visualizer regroups the bins into bands that widen as they rise (an octave, or a third of an octave, per bar), so the display matches the way ears divide up pitch.

Time and frequency trade off. A bigger window gives finer frequency resolution but blurs events in time; a smaller window tracks transients crisply but smears the pitches together. 46 milliseconds is a compromise that suits music. Speech analysers use smaller windows; tuners use enormous ones.

Every spectrum display is a compromise between seeing what is playing and seeing when. The settings that make a kick drum snap make a bass note vague, and the reverse.

Why the bass always wins

Watch any spectrum bar display and the left side dominates. Partly that's the binning issue above, but mostly it's the music. Recorded music carries far more energy in the low frequencies than the high ones; the spectrum of a typical mix slopes downward at roughly 6 dB per octave. The cymbals you hear so clearly are, in raw energy terms, tiny compared with the kick and bass.

Visualizers deal with this in two ways. They show the bars in decibels rather than raw amplitude, which compresses the enormous range into something the eye can read. And many apply a tilt, boosting the display of higher bands, so the picture looks balanced even though the signal isn't. Neither is cheating; both are translations.

The animation is mostly smoothing

If bars simply showed the instantaneous FFT result, they'd flicker unwatchably. So visualizers smooth. Bars rise fast and fall slowly. Peaks hang for a moment before dropping. Values are averaged across a few frames. A beat detector compares the energy in the bass band against its recent average and fires when it spikes, which is what makes shapes pulse in time. Almost everything that feels "musical" about a visualizer's motion is these small choices about decay and hold, layered on top of the raw analysis.

And beyond that, it's art. Once the spectrum and beat information exist, mapping them to colour, particles, geometry and feedback loops is limited only by imagination. MilkDrop's famous look was largely the previous frame being warped, zoomed and blended back into the next one, with the audio nudging the parameters. The analysis is the same in every visualizer; what you see is what the designer did with it.

What it can genuinely tell you

Visualizers are decoration, but a spectrum display is also a real instrument, and it can show you things about your files that your ears might not.

The clearest example is lossy encoding. Encoders save bits by discarding the top of the spectrum, and they do it with a hard cutoff. A 128 kbps MP3 typically loses everything above about 16 kHz; a 320 kbps one keeps up to 19 or 20 kHz. On a spectrum display that appears as a brick wall: bars up to a point, then nothing. A FLAC from a CD shows content all the way to the edge, fading naturally. It's not proof (a lossless file made from a lossy source shows the same wall, which is exactly how transcoded fakes get caught), but it's a fast, honest first check.

A waveform display tells a different story. Put on an older, dynamic master and the shape breathes: peaks and valleys, loud choruses, quiet verses. Put on a loudness-war remaster and it's a solid block that barely moves. You're watching compression, and it's often easier to see than to hear on first listen.

What no visualizer can show is quality in any meaningful sense. Bars to 20 kHz don't mean a good recording; a full waveform doesn't mean a good mix. It shows the signal. Judging the music is still your job.

A note on where it happens

In a browser-based player, all of this runs on your machine, on the decoded audio, after any EQ or effects you've applied. The pixels are computed from the same samples that are going to your headphones; nothing is uploaded or looked up. That's true of the player generally, and it's part of why a visualizer can react instantly: it's a mirror held up to the signal, a few milliseconds behind your ears.

Want to watch a track breathe? Open the player, drop in a FLAC and an MP3 of the same song, and switch visualizers until you find the one that shows the difference.