How a music visualizer works, from sound to surface.

A music visualizer is two programs: one listens, the other draws. This page covers how LIQUIDWAVE listens, from sound to the numbers a scene reads, and the rules for how those numbers may move the picture. The figures come from the app's source.

From samples to a spectrum

The analysis runs once for every frame LIQUIDWAVE draws: 60 times a second by default, up to 120 on a ProMotion display at Full render quality, and 30 in Low Power Mode. Each time, it takes the latest 2,048 samples of mono audio, applies a Hann window and runs a fast Fourier transform (FFT) with Apple's Accelerate framework. At 48 kHz that is 43 ms of sound in bins about 23 Hz wide, so consecutive frames overlap.

The samples come from system audio or from Listen Mode's microphone chain, and both get the same analysis.

Seven bands

The spectrum is split into seven bands:

• Sub-bass, 20–60 Hz
• Bass, 60–250 Hz
• Low-mid, 250–500 Hz
• Mid, 500 Hz–2 kHz
• High-mid, 2–4 kHz
• High, 4–8 kHz
• Air, 8–20 kHz

Each band averages its bins, converts to decibels and maps −60 dB to 0 and 0 dB to 1. It is then smoothed twice, rising faster than it falls: bass takes about 0.15 s to rise and 0.33 s to fall.

Every frame also measures loudness (RMS), brightness (the spectral centroid) and spectral flux: for every bin that got louder since the previous frame, how much it rose, added up.

Onsets: when a band enters

Each band has its own onset detector. It keeps a moving average of the band over about 25 frames and fires when the band jumps above 1.6 times that average and above an absolute floor. The pulse snaps to 1, fades with a 0.1-second time constant, and can't fire again on that band for 180 ms.

Because the average catches up, onsets mark hi-hats arriving over a bass-only intro, not every hat in the groove.

Beats and tempo

A beat is spectral flux above 1.55 times its average over the last 43 frames, above a noise floor, and at least 0.27 s after the previous beat, which caps it near 220 BPM.

The tempo is the median of the last twelve intervals between beats, three bars of 4/4. A median ignores one missed or doubled beat. Scenes also get the beat phase and the time since the last beat.

Kick, snare and hat

Flux says something happened. The drum detector says what, from three slices of the raw spectrum: 35–130 Hz for the kick, 1–2.5 kHz for the crack of a snare or clap, and 3–10 kHz for the hats.

A hit is a slice rising above both its 0.4-second and its 80-millisecond average, and above an absolute level. The second test keeps a swell that takes half a second to build from counting as a drum. A snare and a hat both reach both upper slices, so each is weighed by how much of the rise sits in its own slice.

Each drum becomes an envelope that jumps to the strength of the hit and falls in wall-clock time, 0.16 s for the kick, 0.18 s for the snare and 0.10 s for the hat, so a 60 Hz and a 120 Hz display draw the same hit. Calibrated on six records, the kick lands on the beat grid with 75 to 95 percent of its weight, the snare heaviest on 2 and 4, the hat on the off-beats.

Motion that never runs backward

The beat phase runs from 0 to 1 across each beat, but it is re-fitted: every detected beat restarts it and every new interval rescales it, so it can jump backward mid-beat, and anything positioned by it stutters. The rule came from Curtain, whose bands jittered on beat boundaries until they moved on a count instead.

So position comes from three integrators that only ever climb:

• Beats elapsed adds up the smoothed tempo every frame, or two beats a second without one, so a new estimate changes the speed, never the position.
• Loudness elapsed adds up a slowly smoothed RMS, so motion quickens in a loud passage and slows in a quiet one without reversing.
• Hits elapsed adds up the kick, the snare and half the hat, about one step per full kick.

For geometry that should grow with a passage rather than with each kick, scenes read the bass smoothed over 1.2 s and the centroid over 3 s.

Audio moves the surface, not the light

A picture that brightens on every beat can flash faster than is safe to watch. WCAG 2.3.1 allows three large flashes a second, and a kick with a snare at 130 BPM already passes that. So the drums move things first, through size, displacement, width or a clock that lurches, and change color second.

Where a hit lifts the brightness of the whole frame, the lift stays small and goes through one shared function written for that limit. It compresses stacked hits, and above 180 BPM it lets through only every other beat.

The Metal scenes go further: no beat, onset or drum reaches a color term. Their relief follows the slow bass, and a hit sends a wave across the surface, the one a click on the canvas sends, at a twentieth of its height. While beat terms still reached their color, the chrome read as a flickering screen.

What it doesn't do

It doesn't separate instruments. The drums are slices of one mixed spectrum, so a bright snare also shows in the hat at about half strength, and a loud off-beat hat in the snare.

The tempo is an estimate. The source code calls the detector adequate for visualization and not precise enough for DJ-grade tempo extraction.

It hears one channel. Both sources are mixed to mono before the FFT, so stereo width and panning don't reach the scenes.