Decompose — HPSS, masks, and source separation

decompose operates on spectrograms. Its separation story is blind (unsupervised) source separation: Pleco-Xa’s single canonical harmonic/percussive separation (HPSS), the soft-mask primitive underneath it, and nearest-neighbour filtering for REPET-SIM-style repetition removal — these need nothing but the mixture. Alongside them sits a multi-scale spectral-fingerprinting pipeline for stem-guided spectral matching (supervised — requires reference stems): it learns EQ curves that pull the mixture toward fingerprints of a vocal stem you supply, so it cannot separate a mix it has no reference for. There is no machine-learning model anywhere in this namespace: zero weights, zero inference runtime, no ONNX, just median filters, masks, and gradient-optimised EQ curves.

The HPSS/softmask core was validated against reference fixtures during development (including margin=2); harmonic + percussive ≈ the input at margin=1.

Key functions

Verified against the built barrel (decompose namespace):

• hpss(S, opts) → { harmonic, percussive } spectrograms. Default (mask: false) returns the masked components S · mask, so harmonic + percussive ≈ S at margin=1. Accepts magnitude rows or complex {real, imag} bins (phase is reapplied).
• softmask(X, X_ref, opts) → mask matrix X^p / (X^p + X_ref^p) with rescale-by-max stabilisation; power: Infinity gives a hard mask (X > X_ref).
• nn_filter(S, opts) — replace each frame by an aggregate of its nearest neighbours in a recurrence graph. aggregate: 'median' + metric: 'cosine' + a width band is the REPET-SIM configuration.
• processAudioToFingerprints(audioBuffer, nFft?, hopLength?) — the stem-guided matching entry point: multi-scale spectral fingerprints from an AudioBuffer-shaped input.
• optimizeEqCurves(vocalFps, mixtureFps, mixtureMag, numWindows, sr, ...) — gradient descent for the per-window EQ curves that match the mixture to the reference vocal’s fingerprints (supervised — the optimization target comes from the isolated stem).
• reconstructVocal(mixtureStft, eqCurves, sr, nFft?, hopLength?) → Float32Array of the reconstructed vocal estimate.

Two HPSS entry points, on purpose. decompose.hpss(S, …) takes a spectrogram and returns spectrograms. effects.hpss(y, …) takes a waveform, runs this same core, and inverts back to time-domain signals.

Example

import { decompose, stft } from 'pleco-xa'

// y: Float32Array (mono)
const D = stft(y, 2048, 512) // [freq][time] complex bins

const { harmonic, percussive } = decompose.hpss(D, { margin: 1 })
// harmonic + percussive ≈ D (masked components) at margin=1

// REPET-SIM-style repetition suppression on a magnitude spectrogram S:
const foreground = decompose.nn_filter(S, {
  aggregate: 'median',
  metric: 'cosine',
  width: 3,
})

Notes

• hpss default is masked components, not raw median filters. The default applies the soft masks so the components sum back to the input — you get separated layers, not the bare median-filtered spectrograms. kernel_size default is 31; margins must be >= 1.
• The fingerprint pipeline is supervised — it does not do blind separation. Its optimization target is built from the true isolated vocal, so it needs the reference stem it is matching toward; to separate a mix you only have the mixture of, use hpss/softmask/nn_filter. Input only needs { getChannelData, sampleRate } — it runs in Node against a structural mock, no browser required. optimizeEqCurves/reconstructVocal report progress through the library’s debug logger, silent unless you call setDebug(true); the pipeline’s exact multi-stage wiring (fingerprints → mixture magnitude → EQ curves → reconstruction) is shown end-to-end in the vocal-separation demo.
• nn_filter builds its recurrence graph from segment.recurrenceMatrix; frames with no neighbours pass through unchanged. Supported aggregate values are 'mean', 'median', 'average' (weighted by the graph), or a custom (values, weights) => number.
• NMF decomposition is deliberately out of scope — the one candidate implementation converged incorrectly, so it was left unexported rather than shipping a wrong result.

API reference

Full signatures: decompose namespace — e.g. hpss, softmask, nn_filter, processAudioToFingerprints.