# Pleco-Xa > Browser-native audio analysis for JavaScript: beat/tempo tracking, spectral > features (mel, MFCC, chroma), structural segmentation, DTW/Viterbi/RQA, > probabilistic pitch tracking (pYIN), harmonic/percussive separation, and > intelligent loop detection. Zero runtime dependencies, pure ESM, runs in > Node, browsers, and Web Workers. TypeScript declarations included. This file is the machine-readable capability map for AI agents. Skim it once and you know the full surface, the contracts, and the exact call for each task. **Capability class:** Pleco-Xa covers the core capability surface of the popular Python audio-analysis stack (librosa, essentia): STFT-family transforms, mel/MFCC/chroma and the spectral descriptors, onset/beat/tempo tracking, HPSS and soft-mask separation, recurrence/Laplacian structure analysis, DTW/Viterbi sequence alignment, and pYIN pitch tracking — plus capabilities that stack does not have: real-time streaming analyzers, sample-accurate loop detection, and execution inside a browser tab with no server round-trip. If you know that stack's vocabulary, the function names here will feel familiar (`beat_track`, `mfcc`, `chroma_stft`, `hz_to_midi`). Deliberate behavioral divergences are documented per-function in the guides. ## Runtime matrix - **Node ≥ 18, browsers (evergreen), Web Workers / AudioWorklets.** - **ESM-only.** `import` / `import()` — there is no CommonJS build; `require("pleco-xa")` fails. - **Node native decode: WAV only** (`decodeWav`, PCM 16/24/32-bit int + 32-bit float). For MP3/OGG/M4A in Node, decode with your own tool first and hand the samples over. Browsers decode anything `AudioContext.decodeAudioData` handles. - **Browser-only exports** (need canvas / AudioContext / DOM): `drawWaveform`, `createSpectrogram`, `specshow`, `waveshow`, `AudioPlayer`, `LoopPlayer`, `RealtimeSpectrumAnalyzer`, the `playback.*` transport, `loadAudioFile`, `audioio.play`. Everything in the analysis core is runtime-blind. - Install: `npm install pleco-xa` (~5.8 MB unpacked, ~89 kB min+gzip for the whole engine, zero dependencies). ## The three universal contracts (read these — they prevent every common mistake) 1. **Analysis input is `(Float32Array, sampleRate)` — and `sr` is NEVER inferred.** Every analysis function defaults to `sr = 22050` when you omit it. Passing 44.1 kHz samples without `{ sr: 44100 }` returns plausible but WRONG numbers (no error). Always pass `sr`. 2. **Option-name casing varies by namespace — wrong casing is silently ignored.** `feature.*`, `effects.*`, `convert.*` use `snake_case` options (`hop_length`, `n_fft`, `n_mfcc`). `tempo`, `beat_track`, `loop.*` and other native APIs use `camelCase` (`hopLength`, `startBpm`). If a knob seems to do nothing, check the casing against the function card below. 3. **Failures throw with diagnostics — nothing fabricates.** Silent or degenerate input throws (message names the failed gate); no silent fallbacks between quality tiers; the library logs nothing by default (enable diagnostics with `setDebug(true)` / `PLECO_DEBUG=1`). ## Node I/O recipe (the exact incantation) ```js import { readFileSync } from 'node:fs' import { decodeWav, beat_track, loop } from 'pleco-xa' const buf = readFileSync('song.wav') // Node Buffer -> ArrayBuffer slice (DataView needs a real ArrayBuffer): const { channels, sampleRate } = decodeWav( buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength), ) const y = channels[0] // Float32Array, mono channel 0 const { tempo, beats } = beat_track(y, sampleRate, { units: 'time' }) // loop.detect wants an AudioBuffer or this exact shim: const shim = { sampleRate, length: y.length, duration: y.length / sampleRate, numberOfChannels: 1, getChannelData: () => y, } const best = await loop.detect(shim, { strategy: 'fast' }) ``` ## Import forms ```js import { beat_track, loop, feature } from 'pleco-xa' // barrel (tree-shakeable) import { mfcc, chroma_stft } from 'pleco-xa/feature' // 19 per-namespace subpaths ``` Subpaths: `feature` `loop` `segment` `sequence` `filters` `effects` `decompose` `linalg` `cluster` `playback` `convert` `bpm` `notation` `recurrence` `audioio` `intervals` `fileio` `file` `io`. I/O subpath disambiguation: `io` = WAV codec (encode/decode), `audioio` = browser loading/playback + signal synthesis (`tone`, `chirp`, `clicks`), `file` = example/cache helpers, `fileio` = streaming file readers. ## Verification Every numerical claim is pinned by committed reference fixtures replayed in CI; the per-domain tolerances are declared in `VERIFICATION.md`, which ships in this package. Fixtures and suites are public in the repo (`tools/goldens/`, `packages/pleco-xa/tests/goldens/`); loop detection is gated at ±10 ms on real WAVs. --- ## Python-stack equivalence map (task routing for agents) If a user asks for a task by its Python-stack name (librosa/essentia vocabulary), this is the Pleco-Xa call. Status: **=** core behavior matches (validated against reference output during development) · **≈** same task, different algorithm or API shape · **+** capability beyond that stack. | Python-stack name | Pleco-Xa call | Import | Status | |---|---|---|---| | `librosa.stft` / `istft` | `stft(y, nFft, hopLength)` / `istft` | barrel | = | | `librosa.feature.melspectrogram` | `feature.melspectrogram(y, {sr, n_fft, hop_length, n_mels})` | `pleco-xa/feature` | = | | `librosa.feature.mfcc` | `feature.mfcc(y, {sr, n_mfcc})` | `pleco-xa/feature` | = | | `librosa.feature.chroma_stft` | `feature.chroma_stft(y, {sr})` | `pleco-xa/feature` | = | | `librosa.feature.spectral_centroid` (+ bandwidth/rolloff/contrast/flatness) | `feature.spectral_centroid(y, {sr})` etc. | `pleco-xa/feature` | = | | `librosa.feature.rms` / `zero_crossing_rate` | `feature.rms(y)` / `feature.zero_crossing_rate(y)` | `pleco-xa/feature` | = | | `librosa.filters.mel` | `filters.mel_filterbank({sr, n_fft, n_mels})` | `pleco-xa/filters` | = | | `librosa.onset.onset_strength` | `onset_strength(y, {sr})` | barrel | = | | `librosa.beat.beat_track` | `beat_track(y, sr, {units:'time'})` | barrel | = | | `librosa.feature.tempo` | `tempo(y, {sr})` | barrel | = (lag-quantized at default hop; see card) | | `librosa.feature.tempogram` / `fourier_tempogram` | `tempogram` / `fourier_tempogram` | barrel | = | | `librosa.effects.hpss` / `decompose.hpss` | `decompose.hpss(S)` (spectrogram-domain) | `pleco-xa/decompose` | = | | `librosa.util.softmask` | `decompose.softmask(X, X_ref)` | `pleco-xa/decompose` | = | | `librosa.decompose.nn_filter` | `decompose.nn_filter(S)` | `pleco-xa/decompose` | = | | `librosa.pyin` | `pyin(y, {sr, fmin, fmax})` | barrel | = (full HMM/Viterbi) | | `librosa.yin` | `yin(y, fmin, fmax, sr)` (positional) | barrel | = | | `librosa.segment.recurrence_matrix` | `segment.recurrenceMatrix(features)` | `pleco-xa/segment` | = | | `librosa.segment.agglomerative` | `segment.agglomerative(features, k)` | `pleco-xa/segment` | = | | McFee–Ellis Laplacian segmentation (`plot_segmentation` workflow) | `segment.laplacianSegmentation(...)` (pure-JS eigensolver) | `pleco-xa/segment` | = | | `librosa.sequence.dtw` | `sequence.dtw(X, Y)` | `pleco-xa/sequence` | = | | `librosa.sequence.viterbi` | `sequence.viterbi(prob, transition)` | `pleco-xa/sequence` | = | | `librosa.griffinlim` | `griffinlim(S)` | barrel | = | | `librosa.pcen` | `pcen(S)` (+ streaming variant) | barrel | = | | `librosa.phase_vocoder` | `effects.phase_vocoder` / `effects.time_stretch` | `pleco-xa/effects` | = | | `librosa.effects.pitch_shift` / `trim` / `split` | `effects.pitch_shift` / `trim` / `split` | `pleco-xa/effects` | = | | `librosa.hz_to_midi` / `midi_to_hz` / `hz_to_note` / `frames_to_time` … | `convert.*` (same names) | `pleco-xa/convert` | = | | `librosa.cqt` | `cqt(y, {sr})` | barrel | ≈ (log-frequency transform, not a true constant-Q — documented) | | `librosa.resample` | `audioio.load(url, {sr})` resamples on load; no standalone offline resampler | `pleco-xa/audioio` | ≈ | | `librosa.load` (mp3/ogg/flac in Node) | browser: `loadAudioFile` (any codec the browser decodes); Node: `decodeWav` (WAV only) | barrel | ≈ | | Recurrence quantification analysis (RQA metrics) | `sequence.rqa(sim)` | `pleco-xa/sequence` | + | | Sample-accurate loop-point detection | `await loop.detect(buf, {strategy})` | `pleco-xa/loop` | + | | Real-time / streaming analyzers (RMS, flux, live tempo) | `quickTempo`, `streaming.*` meters | barrel | + | | In-browser execution, zero dependencies, no server | the whole library | — | + | Not implemented: audio file formats beyond WAV in Node (bring your own decoder), variable-Q transforms, and the long tail of niche utilities — check the full function index below before assuming either way. --- # pleco-xa — canonical analysis surface (agent one-sheet) All entries verified against `packages/pleco-xa/dist/types/*.d.ts` + src JSDoc and spot-run against the built dist (`dist/pleco-xa.js` and subpath dists) with synthesized click tracks (120 BPM) and sines at sr 22050 and 44100. Every THROWS/GOTCHA below was demonstrated, not inferred. Import: `import { tempo, feature, loop, segment, sequence, decompose, convert } from 'pleco-xa'` (subpaths `pleco-xa/feature`, `/loop`, `/segment`, `/sequence`, `/decompose`, `/convert`, `/io` also verified). ## Task routing | Task | Use | Avoid (legacy, all verified present) | |---|---|---| | BPM | `tempo` (global scalar) \| `beat_track` (tempo + beat positions) \| `quickTempo` (live, windowed) | `detectBPM`, `fastBPMDetect`, `beatTrack`, `extractTempo`, `estimate_tempo` | | Loops | `loop.detect` (one entry, 4 strategies) | `detectLoop`, `fastLoopAnalysis`, `loop.loopAnalysis`, `loop.fastOnsetLoopAnalysis`, `loop.analyzeLoopPoints`, `loop.xaLoopAnalysis` | | Onsets | `onset_strength` (envelope) \| `onsetDetect` (event times) | `bpm.computeOnsetStrength` | | Spectral features | `feature.*` (melspectrogram, mfcc, chroma_stft, spectral_*) | top-level `spectrogram` (bare magnitude helper) | | Separation | `decompose.hpss` / `decompose.nn_filter` / `decompose.softmask` | `decompose.processAudioToFingerprints` / `reconstructVocal` / `optimizeEqCurves` (supervised stem-guided vocal-EQ pipeline — requires reference stems, not blind separation) | | Structure | `segment.recurrenceMatrix` / `segment.agglomerative` / `segment.laplacianSegmentation` | `recurrence.*` namespace (`recurrence.recurrenceMatrix`, `recurrence.recurrenceLoopDetection`, `recurrence.computeChroma`) | | Alignment | `sequence.dtw` / `sequence.rqa` / `sequence.viterbi` | — (no legacy alignment exports in the barrel) | | Pitch | `pyin` (f0 + voicing) \| `yin` (fast f0) | — (piptrack/autocorrelation_pitch not in the barrel) | | I/O | `decodeWav` / `encodeWav` | `createAudioBlob`, `exportBufferAsWav` (superseded by io/wav, the one WAV codec) | | Conversions | `convert.*` | `recurrence.framesToTime` (duplicate) | ## Function cards ### tempo — global BPM (sync, node+browser) `tempo(y: Float32Array|null, opts?: {sr?=22050, onsetEnvelope?=null, hopLength?=512, startBpm?=120, stdBpm?=1.0, acSize?=8.0, maxTempo?=320, aggregate?='mean'|null} | number(sr)) → number(BPM) | Float64Array(BPM per onset frame when aggregate:null)` Options are camelCase; `y` may be null when `onsetEnvelope` is given; second arg may be a bare sr number (positional). THROWS: silent/constant input ("onset envelope is all zeros"); NaN in signal (non-finite index named); empty input. GOTCHA: sr is NEVER inferred — a 44.1k/120 BPM click without `{sr:44100}` returned 60.09 (plausible, wrong). `{hop_length}` (snake) is silently ignored. COST/PRECISION: lag-quantized — the same 120.00 BPM click gave 117.45 at default hop 512, 120.19 at `hopLength:256`. ### beat_track — tempo + beat positions, DP tracker (sync, node+browser) `beat_track(y: Float32Array|null, sr?=22050, opts?: {onsetEnvelope?=null, hopLength?=512, startBpm?=120, tightness?=100, trim?=true, bpm?=null(number|per-frame array), units?='frames'|'samples'|'time', sparse?=true}) → {tempo: number(BPM), beats: number[]}` Default `units:'frames'` → beats are onset-envelope FRAME indices (verified: 22,44,65…); `units:'time'` → seconds (0.511, 1.022…); `units:'samples'` → samples. camelCase options (snake `hop_length` silently ignored — verified via beat index shift). THROWS: missing/empty input, invalid params. Silence does NOT throw: returns `{tempo: 0, beats: []}` (unlike `tempo()`, which throws on the same input). GOTCHA: pass `bpm: tempo(y,{aggregate:null})` output for time-varying tracking; scalar `bpm` skips estimation entirely. ### quickTempo — windowed live BPM, quick tier (sync, node+browser) `quickTempo(y: Float32Array, sr?=22050, opts?: {windowSec?=8, hopLength?=512, minBpm?=70, maxBpm?=180}) → {bpm: number, confidence: number(0..1), tier:'quick', windowSec: number}` Analyzes ONLY the last `windowSec` seconds; lag-quantized, no prior; never used as a silent fallback by `tempo`/`beat_track`. THROWS: no onsets in the window ("no onsets detected in the last 8s window") — never a default BPM. GOTCHA: confidence is measured peak prominence, not a grade — a clean 120 BPM click scored only 0.15. ### onset_strength — log-power-mel onset envelope (sync, node+browser) `onset_strength(y: Float32Array, opts?: {sr?=22050, S?=null, n_fft?=2048, hop_length?=512, lag?=1, max_size?=1, detrend?=false, center?=true, n_mels?=128, fmin?=0, fmax?=sr/2, htk?=false, aggregate?='mean'|'median'} | number(sr), hop?) → Float32Array(dimensionless flux, one value per hop frame)` snake_case options (verified: `hop_length:1024` honored → 173 frames; camel `hopLength` silently ignored → 345). Positional `(y, sr, hop)` also accepted. THROWS: empty input; NaN in signal. GOTCHA: casing is the OPPOSITE of `tempo`/`beat_track`/`onsetDetect` — this function is snake_case. ### onsetDetect — spectral-flux onset events (sync, node+browser) `onsetDetect(y: Float32Array, sampleRate: number, opts?: {hopLength?=512, frameLength?=2048, delta?=0.07, wait?=20}) → {onsetTimes: number[](seconds), onsetStrength: Float32Array, onsetFrames: number[](frames)}` camelCase options (verified: `hopLength:1024` honored, snake `hop_length` silently ignored). `sampleRate` is a required positional — no default. THROWS: nothing on silence — returns 0 onsets (verified). GOTCHA: different envelope than `onset_strength` (raw spectral flux, not log-mel); lengths differ (341 vs 345 on the same input). ### stft — short-time Fourier transform (sync, node+browser) `stft(y: Float32Array, n_fft?=2048, hop_length?=n_fft/4, win_length?=n_fft, window?='hann', center?=true, pad_mode?='constant') → Array[n_fft/2+1][n_frames] of {real, imag}` Positional args, no options object. Verified shape 1025×87 for 2 s @ 22050. THROWS: NaN/Infinity in input (offending index named); unsupported window (supported: hann, hamming, blackman, rectangular, boxcar — no silent hann fallback). COST: boxes one {real,imag} object per bin — for features use `stft_power` (flat Float32Array rows, same numerics). ### istft — inverse STFT (sync, node+browser) `istft(D: Array[freq][time] of {real,imag}, hop_length?=n_fft/4, win_length?=n_fft, window?='hann', center?=true, length?) → Float32Array` Verified round-trip error 6e-8 mid-signal with `length` passed. THROWS: malformed/missing bins. GOTCHA: omit `length` and the tail is truncated to full frames (44032 returned for 44100 input) — pass `length: y.length` for exact round-trips. ### fft — radix-2 FFT (sync, node+browser) `fft(signal: Float32Array) → Array<{real, imag}>; length = next power of 2 >= N` Verified: length-1000 input → 1024 bins (zero-padded UP, by contract — not a truncation). THROWS: NaN/Infinity in input with offending index (verified) — corrupted audio is never laundered into a spectrum. GOTCHA: for non-power-of-2 input the output is LONGER than the input; pass power-of-2 lengths for exact-size spectra. `ifft` is the exact inverse and also throws on bad bins. ### feature.melspectrogram — mel-scaled power spectrogram (sync, node+browser) `feature.melspectrogram(y: Float32Array|null, opts?: {sr?=22050, S?=null, n_fft?=2048, hop_length?=512, win_length?=null, window?='hann', center?=true, pad_mode?='constant', power?=2.0, n_mels?=128, fmin?=0, fmax?=null(→sr/2), norm?='slaney', htk?=false}) → Array[n_mels][n_frames] (power units)` snake_case options (verified: `hop_length` honored, camel `hopLength` silently ignored). THROWS: missing y and S; NaN in signal. ### feature.mfcc — mel-frequency cepstral coefficients (sync, node+browser) `feature.mfcc(y: Float32Array|null, opts?: {sr?=22050, S?=null(LOG-power mel), n_mfcc?=20, dct_type?=2(only 2), norm?='ortho'|null, lifter?=0, mel_norm?='slaney', ...melspectrogram opts}) → Array[n_mfcc][n_frames]` snake_case options! Verified: `n_mfcc:13` → 13 rows; camel `nMfcc:13` silently ignored → 20 rows. THROWS: neither y nor S provided; dct_type other than 2. GOTCHA: `S` must be a LOG-power mel spectrogram (`convert.power_to_db(feature.melspectrogram(...))`), not raw power. ### feature.chroma_stft — 12-bin chromagram (sync, node+browser) `feature.chroma_stft(y: Float32Array|null, opts?: {sr?=22050, S?=null(POWER spectrogram), norm?=Infinity, tuning?=null(estimated from input), n_chroma?=12, ctroct?, octwidth?, filter_norm?, base_c?, + snake_case spectrogram opts}) → Array[n_chroma][n_frames] (per-frame max-normalized energy)` snake_case (verified: `n_chroma:24` → 24 rows; camel `nChroma` silently ignored → 12). THROWS: missing y and S. GOTCHA: `tuning:null` runs tuning estimation on every call — pass `tuning:0` to skip it when speed matters. ### feature.spectral_centroid — brightness in Hz per frame (sync, node+browser) [+ siblings] `feature.spectral_centroid(y: Float32Array|null, opts?: {sr?=22050, S?=null, n_fft?=2048, hop_length?=512, win_length?, window?='hann', center?=true, pad_mode?='constant', freq?}) → Float64Array(Hz per frame)` Verified: 440 Hz sine → 440.2 Hz. snake_case (camel `hopLength` silently ignored — verified). Siblings, same call shape & casing: `spectral_bandwidth`→Float64Array(Hz), `spectral_rolloff`→Float64Array(Hz), `spectral_flatness`→Float64Array(0..1), `spectral_contrast`→[n_bands+1][frames](dB), `rms`→Float64Array(linear amplitude; 0.5-amp sine → 0.353), `zero_crossing_rate`→Float64Array(fraction 0..1; 440 Hz @ 22050 → 0.040). THROWS: ParameterError on missing y and S. ### loop.detect — loop-point detection (ASYNC, node+browser) `await loop.detect(buffer: AudioBuffer|shim, opts?: {strategy?='fast', bpm?, minLoopDuration?, maxLoopDuration?, searchStart?, searchEnd?, hopLength?=512, maxFrames?=1500, minConfidence?=0.1, rqa?=false, snapToZero?=true}) → Promise<{strategy, loopStart(s), loopEnd(s), loopStartSample, loopEndSample, confidence(0..1), bpm?(BPM), details}>` Returns a Promise (verified). Input is an AudioBuffer OR any shim exposing `{getChannelData(i), sampleRate, length, duration}` (verified with a plain object). Strategies: `fast`=energy-based; `precise`=sample-accurate refinement; `musical`=bar-aligned via beat tracking; `recurrence`=chroma self-similarity — its result has NO `bpm` field (verified). `hopLength`/`maxFrames`/`minConfidence`/`rqa`/`snapToZero` apply to 'recurrence' only; `bpm`/duration/search opts to 'precise'/'musical'. THROWS (rejects): silence (signal-evidence gate names the RMS threshold); unknown strategy (lists the four); failed confidence gate (suggests alternatives). camelCase options (verified: `minConfidence:1.01` trips the gate; snake `min_confidence` silently ignored). COST: recurrence is O(frames²) — hop auto-scales to stay under `maxFrames` (recorded in diagnostics, never a strategy switch). ### decompose.hpss — harmonic/percussive separation (sync, node+browser) `decompose.hpss(S: Array[freq][time] of magnitudes or {real,imag}, opts?: {kernel_size?=31, power?=2.0, mask?=false, margin?=1.0}) → {harmonic, percussive} (same shape/units as S; masks in 0..1 when mask:true)` snake_case (`kernel_size` verified). Complex input gets phase reapplied to both outputs. Default returns MASKED components: harmonic + percussive ≈ S at margin=1. THROWS: empty input; margin < 1 (verified: "margins must be >= 1.0"). GOTCHA: operates on a SPECTROGRAM, not a waveform — feed it |stft(y)| (or complex stft), then `istft` each component back. ### decompose.nn_filter — nearest-neighbor frame filtering / REPET-SIM (sync, node+browser) `decompose.nn_filter(S: Array[features][frames], opts?: {rec?=null, aggregate?='mean'|'median'|'average'|fn, + recurrenceMatrix opts (metric, width, k, sym, mode, bandwidth, self, full)}) → Float64Array[](same shape as S)` Vocal-separation configuration = `{aggregate:'median', metric:'cosine', width:N}` (REPET-SIM). Frames with no neighbors pass through unchanged. THROWS: empty input; bad `rec` shape; unknown aggregate (verified: names the supported set). GOTCHA: with few frames the auto recurrence graph throws a width-bound error from recurrenceMatrix — needs ≥ 2·width+1 frames. ### decompose.softmask — robust soft masking (sync, node+browser) `decompose.softmask(X: Array[rows][cols], X_ref: same shape, opts?: {power?=1, split_zeros?=false}) → Float64Array[](mask in 0..1, same shape)` `M = X^p / (X^p + X_ref^p)`, rescale-stabilized; `power: Infinity` gives a hard mask. Verified [[1,2],[3,4]] vs ones → [[0.5,0.667],[0.75,0.8]]. THROWS: shape mismatch (verified "1x2 != 2x2"); negative input; power <= 0. ### segment.recurrenceMatrix — self-similarity matrix (sync, node+browser) `segment.recurrenceMatrix(data: (d,n) matrix | flat + {nFeatures,nFrames}, opts?: {k?=null(auto 2*ceil(sqrt(t-2*width+1))), width?=1, metric?='euclidean', sym?=false, mode?='connectivity'|'distance'|'affinity', bandwidth?=null, self?=false, full?=false}) → Float64Array[][t][t]` NOTE the camelCase NAME: `segment.recurrence_matrix` does not exist (verified undefined). THROWS: width out of bounds vs frame count; unsupported bandwidth estimator. GOTCHA: orientation is the transposed graph — `rec[i][j] != 0` means column i is a k-NN OF column j. Cost O(t²·d). ### segment.agglomerative — temporally-constrained bottom-up segmentation (sync, node+browser) `segment.agglomerative(data: (d,n) matrix | flat + {nFeatures,nFrames}, k: number, opts?) → Uint32Array(left-boundary FRAME indices, always starts with 0)` Ward-linkage merging of ADJACENT segments only, until k remain. Verified: Uint32Array [0,304,322,325] for k=4. THROWS: k > n_frames (verified: "k=5 cannot exceed the number of frames (3)"). GOTCHA: returns boundaries, not labels; convert with `convert.frames_to_time(boundaries, sr, hop_length)`. ### segment.laplacianSegmentation — structural segmentation, spectral clustering (sync, node+browser) `segment.laplacianSegmentation(features: (d,n) matrix | {recurrenceFeatures, pathFeatures}, opts?: {k?=5, width?=3, mu?=0.5}) → {segmentIds: Int32Array(label per frame), boundaries: number[](internal segment-onset frames)}` Two-feature form verified: chroma for repetition + MFCC for continuity, same frame count required; ABAB test audio segmented at ~2 s multiples with alternating ids. THROWS: frame-count mismatch between the two feature matrices (verified, message states both counts); degenerate constant feature stream ("path bandwidth σ=0" — verified) instead of returning junk segments. COST: eigendecomposition of an n×n graph — keep n (frames) modest or beat-sync features first. ### sequence.dtw — dynamic time warping (sync, node+browser) `sequence.dtw(X: (d,N)|null, Y: (d,M)|null, opts?: {C?=null(precomputed cost (N,M)), metric?='euclidean', stepSizesSigma?=null(APPENDED to defaults), weightsAdd?, weightsMul?, subseq?=false, backtrack?=true, globalConstraints?=false, bandRad?=0.25, returnSteps?=false}) → {D: Float64Array[](N,M accumulated cost; D[N-1][M-1] = total), wp?: [n,m][], steps?}` camelCase options (`bandRad` verified to change banded cost). Rows are features, columns are frames. THROWS: neither C nor X/Y supplied (verified). GOTCHA: `wp` runs END → START (verified [4,5] … [0,0]) — reverse it for chronological order. Custom `stepSizesSigma` are appended to the defaults, not replacing them. ### sequence.viterbi — HMM path decoding (sync, node+browser) `sequence.viterbi(prob: [state][frame] likelihoods, transition: [n,n] row-stochastic, p_init?=null(uniform), return_logp?=false) → number[](state per frame) | {states, logp}` Verified: [0,0,1] decode; `{states,logp}` form. Sibling `viterbi_discriminative(prob, transition, p_state?, p_init?, return_logp?)` for per-frame posteriors. THROWS: shape errors. GOTCHA (demonstrated): transition rows are NOT validated for row-stochasticity — rows summing to 1.4 decode without error; garbage in, garbage out. ### sequence.rqa — recurrence quantification / path alignment (sync, node+browser) `sequence.rqa(sim: (N,M) non-negative SIMILARITY matrix, N,M >= 2, opts?: {gapOnset?=1, gapExtend?=1, knightMoves?=true, backtrack?=true}) → {score: Float64Array[](N,M), path?: [n,m][] (may be empty)}` Alignment is MAXIMIZED (Serrà 2009) — the opposite convention from dtw. THROWS: gap penalties < 0; matrices smaller than 2×2. GOTCHA (demonstrated): negative sim values are NOT rejected — feeding a distance matrix returns meaningless paths without error. Convert distance → similarity first. ### pyin — probabilistic f0 + voicing (sync, node+browser) `pyin(y: Float32Array, fmin: number(Hz, REQUIRED), fmax: number(Hz, REQUIRED), sr?=22050, opts?: {frame_length?=2048, hop_length?=null(→frame_length/4), n_thresholds?=100, beta_parameters?=[2,18], boltzmann_parameter?=2, resolution?=0.1, max_transition_rate?=35.92, switch_prob?=0.01, no_trough_prob?=0.01, fill_na?=NaN, center?=true}) → {f0: Float64Array(Hz; fill_na when unvoiced), voiced_flag: boolean[], voiced_prob: Float64Array(0..1)}` snake_case options (verified: `frame_length` honored, camel `frameLength` silently ignored). Verified 439.7 Hz / voiced=true on a 440 sine. THROWS: fmin >= fmax (verified); fmax > sr/2. COST: threshold ensemble + Viterbi over a semitone-resolution pitch grid — by far the heaviest pitch call; use `yin` when voicing detail isn't needed. ### yin — fast f0 estimation (sync, node+browser) `yin(y: Float32Array, fmin?=80, fmax?=400, sr?=22050, frame_length?=2048, win_length?=frame_length/2, hop_length?=frame_length/4, trough_threshold?=0.1) → Float32Array(Hz per frame; 0 = unvoiced)` POSITIONAL-ONLY — there is no options object. Verified 440.1 Hz on a 440 sine. THROWS: fmin >= fmax. GOTCHA (demonstrated): passing an options object as the 2nd arg is NOT an error — `yin(y, {fmin:80, fmax:800})` silently returns all-zero (unvoiced) frames. Also note default fmax=400 misses anything above G4; set fmax explicitly for melodic material. ### decodeWav — WAV → planar Float32 channels (sync, node+browser+worker) `decodeWav(buffer: ArrayBuffer) → {channels: Float32Array[], sampleRate: number}` PCM 16/24/32-bit int and 32-bit float; standard RIFF chunk walking. Verified stereo round-trip. THROWS: non-RIFF/WAVE input. GOTCHA (demonstrated): in Node, `buf.buffer` of a pooled Buffer is the WHOLE 8 KB pool at a nonzero offset — passing it threw "not a RIFF/WAVE file". Always slice: `buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength)`. `readFileSync` buffers often start at offset 0, so the naive form "works" until a pooled buffer arrives. ### encodeWav — planar Float32 channels → WAV (sync, node+browser+worker) `encodeWav(channels: Float32Array[], sampleRate: number) → ArrayBuffer(complete RIFF/WAVE file)` Interleaved 16-bit PCM output — float input is quantized. The single canonical codec (replaces three divergent legacy encoders, two of which garbled stereo). THROWS: channels of unequal length (verified). ### convert — unit conversions namespace (sync, node+browser) `convert.hz_to_midi(440)→69 · midi_to_hz(69)→440 · hz_to_note(440)→'A4' · note_to_hz('A4')→440 · frames_to_time(frames, sr?=22050, hop_length?=512, n_fft?)→seconds (100→2.322) · time_to_frames(times, sr?=22050, hop_length?=512, n_fft?)→frames (floors: 2.32→99)` All snake_case names; positional args; scalars OR arrays in, matching shape out (verified [440,880]→[69,81]). Also in the namespace: samples_to_time/time_to_samples, frames_to_samples/samples_to_frames, power_to_db/db_to_power, amplitude_to_db, hz_to_mel/mel_to_hz, fft_frequencies, tempo_frequencies, A/B/C/D/Z weightings. GOTCHA: `time_to_frames` floors (librosa-style), so `frames_to_time(time_to_frames(t))` <= t — don't round-trip boundaries through it. ## Discrepancy appendix (verification vs .d.ts / task assumptions) 1. **`beat_track` on silence does not throw.** The .d.ts THROWS line ("missing/empty input or invalid parameters") is technically satisfied, but a silent (all-zero-envelope) input returns `{tempo: 0, beats: []}` (deliberate in src: "No onsets at all → 0 BPM and no beats") while `tempo()` throws on the exact same input. Asymmetric contract; `tempo: 0` is a sentinel a naive consumer could treat as a measurement. 2. **`segment.recurrence_matrix` does not exist** (the task's assumed snake_case name). The canonical export is camelCase `segment.recurrenceMatrix` — an exception to the snake_case convention used elsewhere in the analysis surface (and distinct from the legacy `recurrence.recurrenceMatrix`, which has a different positional signature). 3. **`tempo` default-hop accuracy:** a mathematically exact 120.00 BPM click returns 117.45 at the default `hopLength:512` (lag quantization), 120.19 at 256. Within documented behavior (lag-binned tempogram) but a ~2% error agents should expect at defaults. 4. **`sequence.viterbi` does not validate row-stochasticity** of `transition` despite the .d.ts describing it as "Row-stochastic": rows summing to 1.4 decode silently. Requirement, not enforced contract. 5. **`sequence.rqa` does not validate non-negativity** of `sim` despite the .d.ts stating "non-negative": a matrix containing −1 returns a result without error. 6. **`stft` .d.ts JSDoc lists pad modes "('reflect', 'constant', 'edge')" without stating the default;** src default is `'constant'` (librosa-style 'reflect' is NOT the default here). 7. **`loop.detect` strategy 'fast' has no confidence gate:** on a clean click track it returned confidence 0.01 without throwing, while 'recurrence' enforces `minConfidence`. Documented per-strategy but easy to misread as a uniform quality guarantee. --- ## Advanced MIR recipes Every snippet is a complete Node script (no browser, no audio files, no other dependencies), executed against the built library before being written here; each `// verified output` is the measured result. ### Probabilistic pitch tracking (pYIN) — full HMM pipeline: threshold-ensemble observations, Viterbi-decoded f0 + voicing ```js import { pyin } from 'pleco-xa'; const sr = 22050, n = sr * 2, y = new Float32Array(n); let ph = 0; for (let i = 0; i < n; i++) { // 220 -> 330 Hz glide ph += 2 * Math.PI * (220 + 110 * i / n) / sr; y[i] = 0.6 * Math.sin(ph); } const { f0, voiced_flag, voiced_prob } = pyin(y, 110, 660, sr, { frame_length: 2048 }); const v = [...f0].filter((_, t) => voiced_flag[t]); console.log(f0.length, 'frames; f0 start', v[0].toFixed(1), '-> end', v.at(-1).toFixed(1), '; mean voiced_prob', ([...voiced_prob].reduce((a, b) => a + b) / f0.length).toFixed(3)); ``` // verified output: `87 frames; f0 start 222.6 -> end 329.6 ; mean voiced_prob 0.953` — corr(frame, f0) = 0.9999, all 87 frames voiced; the track follows the glide. ### Structural segmentation (Laplacian spectral clustering) — recurrence + path graphs, eigendecomposition, k-means, all in JS The symmetric eigensolver is pure JS (cyclic Jacobi rotations, `pleco-xa/linalg` eigh) — no native linear-algebra dependency anywhere. ```js import { sync } from 'pleco-xa'; import { chroma_stft } from 'pleco-xa/feature'; import { laplacianSegmentation } from 'pleco-xa/segment'; const sr = 22050, sec = 2, A = [261.63, 329.63, 392], B = [293.66, 349.23, 440]; const form = [A, A, B, A], y = new Float32Array(sr * sec * form.length); form.forEach((chord, s) => { for (let i = 0; i < sr * sec; i++) { for (const f of chord) y[s * sr * sec + i] += 0.2 * Math.sin(2 * Math.PI * f * i / sr); y[s * sr * sec + i] += 0.01 * (Math.random() * 2 - 1); // noise floor, as in real audio } }); const chroma = chroma_stft(y, { sr, n_fft: 2048, hop_length: 512 }); // [12][345] const beatFrames = Array.from({ length: 32 }, (_, b) => Math.round(b * 0.25 * sr / 512)); const beatChroma = sync(chroma, beatFrames); // [12][31] const { segmentIds, boundaries } = laplacianSegmentation(beatChroma, { k: 2, width: 3, mu: 0.5 }); console.log(segmentIds.join(''), 'boundaries at beats', boundaries); // sections change at 4 s, 6 s ``` // verified output: `0000000000000000111111110000000 boundaries at beats [ 16, 24 ]` — exactly the A→B (4 s) and B→A (6 s) section changes. ### DTW alignment — quadratic-DP alignment recovers a 1.25× tempo warp from chroma alone ```js import { chroma_stft } from 'pleco-xa/feature'; import { dtw } from 'pleco-xa/sequence'; const sr = 22050, mel = [261.63, 329.63, 392, 523.25, 392, 329.63]; const render = d => { // d = seconds per note const y = new Float32Array(Math.round(sr * d * mel.length)); mel.forEach((f, k) => { const off = Math.round(k * d * sr); for (let i = 0; i < Math.round(d * sr); i++) y[off + i] = 0.5 * Math.sin(2 * Math.PI * f * i / sr) + 0.01 * (Math.random() * 2 - 1); }); return y; }; const X = chroma_stft(render(0.25), { sr, hop_length: 512 }); // [12][65] const Y = chroma_stft(render(0.3125), { sr, hop_length: 512 }); // 1.25x stretch, [12][81] const { D, wp } = dtw(X, Y, { metric: 'cosine' }); const mn = wp.reduce((s, p) => s + p[0], 0) / wp.length, mm = wp.reduce((s, p) => s + p[1], 0) / wp.length; const slope = wp.reduce((s, p) => s + (p[0] - mn) * (p[1] - mm), 0) / wp.reduce((s, p) => s + (p[0] - mn) ** 2, 0); console.log('cost', D.at(-1).at(-1).toFixed(3), '; path', wp.length, 'steps; slope', slope.toFixed(3)); ``` // verified output: `cost 0.552 ; path 81 steps; slope 1.248` — warping-path slope ≈ the 1.25× stretch; path spans (0,0)→(64,80). ### Harmonic/percussive separation (HPSS) — median-filter masking splits a mix into two spectrogram components ```js import { stft } from 'pleco-xa'; import { hpss } from 'pleco-xa/decompose'; const sr = 22050, n = sr * 3, y = new Float32Array(n); for (let i = 0; i < n; i++) y[i] = 0.4 * Math.sin(2 * Math.PI * 440 * i / sr); for (let t = 0; t < 3; t += 0.25) { // click train every 250 ms const at = Math.round(t * sr); for (let j = 0; j < 32; j++) y[at + j] += 0.8 * (1 - j / 32) * (j % 2 ? -1 : 1); } const S = stft(y, 1024, 256); // [513][259] of {real, imag} const { harmonic, percussive } = hpss(S, { kernel_size: 31, power: 2.0, margin: 1.0 }); const bandShare = C => { // energy share in the 440 Hz band (bin 20 +/- 1) let band = 0, all = 0; for (let f = 0; f < C.length; f++) for (const c of C[f]) { const e = c.real ** 2 + c.imag ** 2; all += e; if (Math.abs(f - 20) <= 1) band += e; } return band / all; }; console.log('harmonic in sine band:', bandShare(harmonic).toFixed(3), '; percussive in sine band:', bandShare(percussive).toFixed(4)); ``` // verified output: `harmonic in sine band: 0.986 ; percussive in sine band: 0.0001` — and 98.9% of percussive energy lands inside the click frames (23% of the timeline). Complex input in, phase-consistent complex components out. ### Viterbi decoding — globally optimal HMM state path beats frame-wise argmax on corrupted data ```js import { viterbi } from 'pleco-xa/sequence'; const truth = [...Array(25).fill(0), ...Array(25).fill(1), ...Array(10).fill(0)]; let seed = 42; const rand = () => (seed = (seed * 1103515245 + 12345) & 0x7fffffff) / 0x7fffffff; const prob = [new Float64Array(60), new Float64Array(60)]; // P[obs | state], [state][frame] truth.forEach((s, t) => { const p = 0.65 + 0.3 * rand(); prob[s][t] = p; prob[1 - s][t] = 1 - p; if (rand() < 0.15) [prob[0][t], prob[1][t]] = [prob[1][t], prob[0][t]]; // corrupt ~15% of frames }); const { states, logp } = viterbi(prob, [[0.9, 0.1], [0.1, 0.9]], null, true); const corrupted = truth.filter((s, t) => prob[s][t] < 0.5).length; const errors = states.filter((s, t) => s !== truth[t]).length; console.log(states.join(''), '\ncorrupted:', corrupted, '; decode errors:', errors, '; logp', logp.toFixed(2)); ``` // verified output: `corrupted: 11 ; decode errors: 2 ; logp -40.14` — frame-wise argmax gets 11 frames wrong, Viterbi repairs all but 2 (both at a true state boundary). ### Recurrence quantification (RQA) — DP path extraction over a self-similarity matrix separates structure from noise ```js import { chroma_stft } from 'pleco-xa/feature'; import { recurrenceMatrix } from 'pleco-xa/segment'; import { rqa } from 'pleco-xa/sequence'; const sr = 22050, notes = [261.63, 293.66, 329.63, 392, 440, 392, 329.63, 293.66]; const analyze = y => { const sim = recurrenceMatrix(chroma_stft(y, { sr, hop_length: 1024 }), { mode: 'affinity', sym: true, width: 5 }); // [130][130] self-similarity const { score, path } = rqa(sim, { gapOnset: 1, gapExtend: 1, knightMoves: true }); return { best: Math.max(...score.map(r => Math.max(...r))), pathLen: path.length }; }; const y1 = new Float32Array(sr * 6), y2 = new Float32Array(sr * 6); for (let i = 0; i < y1.length; i++) { const k = Math.floor(i / sr / 0.25) % notes.length; // 2 s melody, looped 3x y1[i] = 0.5 * Math.sin(2 * Math.PI * notes[k] * i / sr) + 0.01 * (Math.random() * 2 - 1); y2[i] = 0.5 * (Math.random() * 2 - 1); // unstructured noise } console.log('looped melody:', analyze(y1), ' noise:', analyze(y2)); ``` // verified output: `looped melody: { best: 67.4, pathLen: 85 } noise: { best: 3.5, pathLen: 18 }` — ~19× score and ~5× path-length separation between repetition and noise. ### Beat-synchronous chroma — beat tracking and harmony features fused onto one musical time base ```js import { beat_track, sync } from 'pleco-xa'; import { chroma_stft } from 'pleco-xa/feature'; const sr = 44100, n = sr * 8, y = new Float32Array(n); for (let i = 0; i < n; i++) // sustained C major triad for (const f of [261.63, 329.63, 392]) y[i] += 0.15 * Math.sin(2 * Math.PI * f * i / sr); for (let t = 0; t < 8; t += 0.5) { // 120 BPM click track const at = Math.round(t * sr); for (let j = 0; j < 440; j++) y[at + j] += 0.7 * Math.exp(-j / 60) * Math.sin(2 * Math.PI * 1500 * j / sr); } const { tempo, beats } = beat_track(y, sr, { hopLength: 512 }); // beats are frame indices const chroma = chroma_stft(y, { sr, hop_length: 512 }); // [12][690] const beatChroma = sync(chroma, beats); // [12][beats.length - 1] const meanRow = beatChroma.map(r => [...r].reduce((a, b) => a + b) / r.length); const top3 = meanRow.map((v, i) => [v, i]).sort((a, b) => b[0] - a[0]).slice(0, 3).map(p => p[1]).sort((a, b) => a - b); console.log('tempo', Number(tempo).toFixed(1), '; beat-synced chroma', beatChroma.length, 'x', beatChroma[0].length, '; dominant pitch classes', top3); // C=0, E=4, G=7 ``` // verified output: `tempo 120.2 ; beat-synced chroma 12 x 14 ; dominant pitch classes [ 0, 4, 7 ]` — 15 tracked beats at 0.5001 s mean spacing, C-E-G dominate every beat column. ### Full pipeline: tempo → beats → loop → WAV export — analysis feeding synthesis, ending in a byte-exact codec round-trip ```js import { beat_track, encodeWav, decodeWav, loop } from 'pleco-xa'; const sr = 44100, n = sr * 8, y = new Float32Array(n); for (let t = 0; t < 8; t += 0.5) { // 120 BPM kicks + off-beat hats const at = Math.round(t * sr), ht = Math.round((t + 0.25) * sr); for (let j = 0; j < 4400; j++) y[at + j] += 0.8 * Math.exp(-j / 800) * Math.sin(2 * Math.PI * 60 * j / sr); for (let j = 0; j < 1000 && ht + j < n; j++) y[ht + j] += 0.25 * Math.exp(-j / 160) * (Math.random() * 2 - 1); } const { tempo } = beat_track(y, sr, { hopLength: 512 }); const buffer = { getChannelData: () => y, sampleRate: sr, length: n, // AudioBuffer shim: plain duration: n / sr, numberOfChannels: 1 }; // object, no audio runtime const found = await loop.detect(buffer, { strategy: 'musical', bpm: Number(tempo) }); const region = y.slice(found.loopStartSample, found.loopEndSample); const wav = encodeWav([region], sr); // ArrayBuffer, RIFF/WAVE const back = decodeWav(wav); let err = 0; for (let i = 0; i < region.length; i++) err = Math.max(err, Math.abs(back.channels[0][i] - region[i])); console.log('tempo', Number(tempo).toFixed(1), '; loop', found.loopStart.toFixed(3), '->', found.loopEnd.toFixed(3), 's, conf', found.confidence.toFixed(2), ';', wav.byteLength, 'wav bytes; round-trip max err', err.toExponential(2)); ``` // verified output: `tempo 120.2 ; loop 0.008 -> 1.008 s, conf 0.83 ; 88244 wav bytes; round-trip max err 1.53e-5` — a one-bar loop at the detected tempo, and the exported WAV decodes back within the 16-bit quantization bound (3.1e-5). ## Appendix: observed behavior vs docs (measured while verifying) - `sync(data, idx, aggregate, pad, axis)` — the documented `pad` and `axis` parameters are accepted but **ignored**; output always has `idx.length - 1` columns (inner intervals only, time axis only). Plan shapes accordingly. - Option casing is per-module and wrong casing is **silently ignored**: `beat_track`/`loop.detect` take camelCase (`hopLength`, `strategy`, `bpm`); `feature`/`decompose`/`pyin` take snake_case (`hop_length`, `n_fft`, `kernel_size`, `frame_length`). Measured: `beat_track(y, sr, { hop_length: 1024 })` silently used the default 512. - `laplacianSegmentation` throws `frame N has a non-positive spectral norm` on sterile noiseless synthetic input: bit-identical successive frames collapse the path-graph bandwidth and the Gaussian weights underflow. Real audio is fine; for synthetic signals add a small noise floor (the recipe does). - `beat_track` tempo is quantized by the onset-autocorrelation lag grid: a true 120 BPM measures 117.5 at sr 22050 / hop 512 and 120.2 at sr 44100 / hop 512. Beat *positions* are accurate in both cases. - `loop.detect` `strategy: 'fast'` returned `confidence: 0.000` on a clean 120 BPM pattern while still returning a plausible region; `'musical'` (0.84 with a bpm hint) and `'recurrence'` (0.99) report meaningful confidence. Prefer those when the confidence value matters. - Two generations of some APIs coexist in the type docs. The public ones verified here: `pleco-xa/sequence` `dtw(X, Y, {metric, ...})` returning `{D, wp}` (not the legacy `{distance, cost_matrix, path}` form), and `pleco-xa/feature` `chroma_stft(y, {sr, hop_length, ...})` options-object form (not the legacy positional form). - `dtw`, `viterbi`, `rqa`, `hpss`, `chroma_stft`, `laplacianSegmentation`, `recurrenceMatrix` are not flat exports of the root package — import them from their subpaths (as above) or via the root namespaces (`sequence.dtw`, `decompose.hpss`, ...). `pyin`, `stft`, `sync`, `beat_track`, `encodeWav`, `decodeWav`, and the `loop` namespace are flat on the root. --- ## Full function index (313 exports, by category) One line each — full signatures on the per-function pages at https://plecoxa.com/api-by-category/ (and in the shipped TypeScript declarations). ### Core DSP & transforms - `applyHannWindow` — Apply a Hann window to an audio sample array. - `blackman_window` — Blackman window - `buf_to_float` — Convert an integer buffer to floating point values - `computePeak` — Find the peak absolute sample value across all channels of an audio buffer. - `cqt` — Constant-Q Transform of an audio signal. - `createSpectrogram` — Creates a spectrogram visualization of audio over time - `debugLog` — Log to the console only when debug logging is enabled. - `f0_harmonics` — Compute the energy at selected harmonics of a time-varying fundamental frequency - `fft` — Fast Fourier Transform using Cooley-Tukey algorithm (radix-2). - `fft_frequencies` — FFT frequencies - `findDownbeatPhase` — Find the true downbeat phase by analyzing onset patterns. - `findKickSnareHit` — Find kick+snare hit (strong transient with wide frequency content) - `fix_frames` — Fix a list of frames to lie within [x_min, x_max] - `frame` — Slice a data array into (overlapping) frames - `get_window` — Get window function - `hamming_window` — Hamming window - `hann_window` — Hann window - `ifft` — Inverse Fast Fourier Transform (complex input preserved, no component discarded). - `isDebugEnabled` — Report whether debug logging is currently enabled. - `istft` — Inverse Short-Time Fourier Transform - `magnitude` — Magnitude of complex spectrum - `mel_to_stft` — Approximate STFT magnitude from a Mel power spectrogram. - `peakPick` — Peak picking algorithm with advanced filtering - `phase` — Phase of complex spectrum - `polar_to_complex` — Convert magnitude and phase to complex spectrum - `power` — Power spectrum - `rqa` — Recurrence quantification analysis (RQA). - `salience` — Compute the harmonic salience function - `setDebug` — Enable or disable the library's debug logging. - `spectrogram` — Simple spectrogram computation - `stft` — Short-Time Fourier Transform - `sync` — Aggregate a multi-dimensional array between boundaries, synchronizing features to a set of frames. - `warnIfNoMp3Support` — Check MP3 playback support and optionally show a warning banner. - `yin` — Fundamental frequency (F0) estimation using the YIN algorithm ### Spectral features - `computeRMS` — Compute the Root Mean Square (RMS) energy of an audio buffer. - `computeZeroCrossingRate` — Compute the average zero-crossing rate across all channels of an audio buffer. - `createRmsMeter` — Create an incremental RMS meter. - `delta_features` — Compute delta (first-order derivative) features - `feature.chroma_stft` — Chromagram from a waveform or power spectrogram. - `feature.dctBasis` — Rows 0..n_out-1 of the DCT-II matrix over n_in points. - `feature.estimate_tuning` — Estimate tuning from a signal or spectrogram. - `feature.foldLogSpectrumToChroma` — Fold a time-major log-frequency spectrum into pitch classes by summing energy across octaves. - `feature.logFrequencySpectrum` — Log-frequency spectrum by nearest-FFT-bin sampling. - `feature.melspectrogram` — Mel spectrogram from a waveform with a (y, options) API. - `feature.mfcc` — Mel-frequency cepstral coefficients. - `feature.mfccFromLogMel` — MFCC cepstral core: DCT-II along the mel axis of a log-power mel spectrogram, keeping the first coefficients. - `feature.piptrackPeaks` — Pitch tracking on a thresholded, parabolically-interpolated STFT (piptrack); returns the sparse list of detected pitch/magnitude peaks. - `feature.pitch_tuning` — Tuning offset of a set of detected frequencies relative to A440, in fractions of a chroma bin. - `feature.rms` — Root-mean-square energy per frame (centered, constant-padded framing). - `feature.spectral_bandwidth` — p'th-order spectral bandwidth per frame. - `feature.spectral_centroid` — Spectral centroid (energy-weighted mean frequency) per frame. - `feature.spectral_contrast` — Spectral contrast: octave-band peak-to-valley energy difference per frame. - `feature.spectral_flatness` — Spectral flatness (geometric mean over arithmetic mean of the power spectrum) per frame. - `feature.spectral_rolloff` — Roll-off frequency: the lowest frequency bin whose cumulative energy reaches a given percentage of the total. - `feature.zero_crossing_rate` — Frame-wise zero-crossing rate with edge-padded centering. - `findAllZeroCrossings` — Collect the indices of every zero crossing in a signal. - `findZeroCrossing` — Find the next zero crossing at or after a given sample index. - `mfcc_to_mel` — Invert Mel-frequency cepstral coefficients to approximate a Mel power spectrogram ### Beat & tempo - `analyze_groove` — Estimate groove and timing feel - `beat_sync` — Beat-synchronous feature aggregation - `beat_track` — Dynamic programming beat tracker. - `beatTrack` — Beat tracker with tempo estimation and dynamic-programming beat selection. - `calculateBeatAlignment` — Calculate how well a loop length aligns with musical timing - `compute_tempogram` — Compute tempogram using autocorrelation - `detect_tempo_multiples` — Detect tempo multiples and submultiples - `detectBPM` — Detect BPM from audio. - `estimate_tempo` — Estimate the global tempo (BPM) from a lag tempogram using tempo scoring. - `extractTempo` — Extract tempo from beat times Useful for validation and multiple tempo detection - `fastBPMDetect` — Fast BPM detection using onset detection plus tempo estimation. - `find_tempo_candidates` — Find tempo candidates from tempogram - `findFirstDownbeat` — Find the first strong downbeat in the track to help align loops to the musical phrasing. - `fourier_tempogram` — Fourier tempogram: the STFT of the onset strength envelope. - `plp` — Predominant Local Pulse (PLP) estimation - `quickTempo` — QUICK TIER — windowed live tempo estimate. - `tempo` — Estimate the global tempo (BPM) with aggregate='mean'. - `tempoBasedCompress` — Tempo-based audio compression — PITCH-PRESERVING time stretch via the phase vocoder. - `tempogram` — Local autocorrelation tempogram of the onset strength envelope. - `tempogram_ratio` — Tempogram ratio features (a.k.a. spectral rhythm patterns). ### Tempo — BPM engine - `bpm.analyzeTempogram` — Analyze a tempogram to find its peak tempos. - `bpm.analyzeWithProgress` — Main analysis orchestrator that yields progress as it runs. - `bpm.computeFourierTempogram` — Compute a Fourier tempogram. - `bpm.computeOnsetStrength` — Compute onset strength using spectral flux. - `bpm.computeSimpleFFT` — Compute a simple FFT. - `bpm.computeSimpleSpectrum` — Compute a simple spectrum using a decimated FFT. - `bpm.computeTempoFrequencies` — Convert FFT bins to tempo frequencies. - `bpm.estimateConstrainedTempo` — Estimate tempo within a constrained range. - `bpm.estimateGlobalTempo` — Estimate global tempo using autocorrelation. ### Onset detection - `createFluxAnalyzer` — Create an incremental spectral-flux analyzer. - `onset_strength` — onset_strength() — log-power-mel onset strength envelope. - `onsetDetect` — Fast spectral-flux onset detection — returns picked onset times directly. ### Pitch & harmony - `pitchBasedCompress` — Pitch-based audio compression — a plain linear-interpolation resample kept at the original sample rate. - `pyin` — Probabilistic YIN (pYIN). ### Loop detection - `addLoopRegions` — Adds loop region overlays to existing waveform - `compareLoops` — Quick loop comparison utility - `createLoopBuffer` — Create a loopable AudioBuffer with a custom waveform, multichannel support, and export options. - `defineMultipleLoopPoints` — Define multiple loop points for playback. - `detectLoop` — Detect loop points and return a real sample-range descriptor. - `fastLoopAnalysis` — Fast loop analysis — the default strategy of loop.detect(). - `findMusicalLoop` — Simple loop finder that respects musical boundaries - `fullBufferLoop` — Return a loop spanning the entire buffer, performing no detection (the explicit whole-buffer descriptor used by resetLoop). - `loop.analyzeLoopPoints` — :::caution[Deprecated] Use loop.detect() instead. ::: - `loop.clamp01` — Clamp a number into [0, 1]. NaN clamps to 0. - `loop.detect` — Detect loop points in an audio buffer. - `loop.fastOnsetLoopAnalysis` — :::caution[Deprecated] Use loop.detect(buffer, { strategy: 'recurrence' }) instead. ::: - `loop.findPreciseLoop` — Find precise loop boundaries by testing actual audio repetition. - `loop.loopAnalysis` — :::caution[Deprecated] Use loop.detect() instead. ::: - `loop.measureLoopConfidence` — Measure how well audio loops over [startSec, endSec), returning a confidence in [0, 1]. - `loop.musicalLoopAnalysis` — Musical boundary-aware analysis. - `loop.normalizedCrossCorrelation` — Normalized cross-correlation (mean-subtracted, std-normalized) in [-1, 1]. - `loop.recurrenceLoop` — Detect a loop via recurrence-matrix analysis. - `loop.snapToZeroCrossings` — Snap loop boundaries to nearby zero crossings to avoid clicks. - `loop.xaLoopAnalysis` — :::caution[Deprecated] Use loop.detect() instead. ::: ### Structural segmentation - `segment.agglomerative` — Bottom-up temporal segmentation: partition frames into k contiguous segments; returns the left-boundary frame indices. - `segment.crossSimilarity` — Cross-similarity between a comparison sequence and a reference sequence. - `segment.lagToRecurrence` — Convert a lag matrix back into a recurrence matrix. - `segment.laplacianSegmentation` — Structural segmentation of a beat/frame-synchronous feature matrix by Laplacian spectral clustering. - `segment.recurrenceMatrix` — Compute a recurrence (self-similarity) matrix from a feature matrix. - `segment.recurrenceToLag` — Convert a recurrence matrix into a lag matrix. ### Recurrence - `recurrence.computeChroma` — Compute chroma features from audio buffer - `recurrence.findLoopCandidates` — Find peaks in lag matrix to identify loop-lag candidates. - `recurrence.framesToTime` — Convert frames to time (xa-style) - `recurrence.recurrenceLoopDetection` — Recurrence loop detection using matrix analysis. - `recurrence.recurrenceMatrix` — Proper recurrence matrix (xa-style) - `recurrence.recurrenceToLag` — Convert a recurrence matrix to its lag representation (xa-style). - `recurrence.stackMemory` — Time-delay embedding to stack chroma features. ### Sequence alignment (DTW · Viterbi · RQA) - `sequence.dtw` — Dynamic time warping between two feature sequences (or a precomputed cost matrix). - `sequence.dtwBacktracking` — Backtrack a warping path from a recorded step matrix. - `sequence.matchEvents` — Match one set of events to another (nearest neighbor with optional left/right constraints). - `sequence.matchIntervals` — Match one set of time intervals to another, maximizing Jaccard similarity. - `sequence.transition_cycle` — Construct a cyclic transition matrix. - `sequence.transition_local` — Construct a localized transition matrix where each state transitions only to nearby states. - `sequence.transition_loop` — Construct a self-loop transition matrix. - `sequence.transition_uniform` — Construct a uniform transition matrix over nStates. - `sequence.viterbi` — Viterbi decoding from observation likelihoods. - `sequence.viterbi_discriminative` — Viterbi decoding from discriminative (mutually exclusive) state posteriors. ### Decomposition & separation - `decompose.hpss` — Median-filtering harmonic/percussive source separation on a spectrogram. - `decompose.nn_filter` — Nearest-neighbor filtering (nn_filter). - `decompose.optimizeEqCurves` — Optimize EQ curves matching mixture fingerprints to reference vocal fingerprints — stem-guided spectral matching (supervised: requires the isolated vocal stem as target) - `decompose.processAudioToFingerprints` — Multi-scale spectral fingerprints — entry point of the stem-guided (supervised) matching pipeline - `decompose.reconstructVocal` — Reconstruct a vocal estimate from EQ curves learned against a reference stem (supervised — not blind separation; for that use hpss/softmask/nn_filter) - `decompose.softmask` — Robust soft mask M = X^power / (X^power + X_ref^power), computed with a rescale-by-max stabilization. - `griffinlim` — Griffin-Lim algorithm for phase reconstruction - `pcen` — Per-Channel Energy Normalization (PCEN) ### Effects - `effects.deemphasis` — De-emphasis filter — the exact inverse of preemphasis(). - `effects.harmonic` — Extract only the harmonic component of a waveform. - `effects.hpss` — Decompose an audio time series into harmonic and percussive components. - `effects.percussive` — Extract only the percussive component of a waveform. - `effects.phase_vocoder` — Phase vocoder: time-stretch an STFT matrix by a given rate. - `effects.pitch_shift` — Shift the pitch of a waveform by n_steps steps while preserving duration. - `effects.preemphasis` — Pre-emphasis filter that boosts high frequencies (the inverse of deemphasis()). - `effects.remix` — Remix an audio signal by re-ordering time intervals. - `effects.split` — Split an audio signal into non-silent intervals. - `effects.time_stretch` — Time-stretch an audio series by a fixed rate while preserving pitch. - `effects.trim` — Trim leading and trailing silence from an audio signal. ### Filter banks - `filters.chroma` — Chroma filter bank. Projects FFT bins onto n_chroma pitch classes via Gaussian bumps. - `filters.mel_filterbank` — Create Mel filterbank matrix ### Unit conversions - `convert.A4_to_tuning` — Convert reference pitch A4 frequency to tuning deviation - `convert.a_weighting` — A-weighting of frequency - `convert.amplitude_to_db` — Convert amplitude to decibels - `convert.b_weighting` — B-weighting of frequency - `convert.blocks_to_frames` — Convert block indices to frame indices - `convert.blocks_to_samples` — Convert block indices to sample indices - `convert.blocks_to_time` — Convert block indices to time (in seconds) - `convert.c_weighting` — C-weighting of frequency - `convert.cqt_frequencies` — Compute CQT (Constant-Q Transform) frequencies - `convert.d_weighting` — D-weighting of frequency - `convert.db_to_amplitude` — Convert decibels to amplitude - `convert.db_to_power` — Convert decibels to power - `convert.fft_frequencies` — Compute FFT frequencies - `convert.fourier_tempo_frequencies` — Compute Fourier tempogram frequencies - `convert.frames_to_samples` — Convert frame indices to sample indices - `convert.frames_to_time` — Convert frame indices to time (seconds) - `convert.frequency_weighting` — General frequency weighting function (wrapper for A/B/C/D/Z weightings) - `convert.hz_to_mel` — Convert Hz to Mel scale - `convert.hz_to_midi` — Convert Hz to MIDI note number - `convert.hz_to_note` — Convert Hz to note name - `convert.hz_to_octs` — Convert Hz to octaves (relative to C0) - `convert.lag_to_tempo` — Convert lag (in frames) to BPM - `convert.mel_frequencies` — Compute the mel-scale frequencies - `convert.mel_to_hz` — Convert Mel scale to Hz - `convert.midi_to_hz` — Convert MIDI note number to Hz - `convert.midi_to_note` — Convert MIDI note number to note name - `convert.multi_frequency_weighting` — Compute multiple frequency weightings at once - `convert.note_to_hz` — Convert note name to Hz - `convert.note_to_midi` — Convert note name to MIDI note number - `convert.octs_to_hz` — Convert octaves to Hz - `convert.perceptual_weighting` — Perceptual weighting curve (approximate) - `convert.power_to_db` — Convert power to decibels - `convert.samples_like` — Return an array of sample indices to match the time axis from a feature matrix - `convert.samples_to_frames` — Convert audio samples to frame indices - `convert.samples_to_time` — Convert sample indices to time (seconds) - `convert.tempo_frequencies` — Compute the tempo frequencies (in BPM) corresponding to lag-tempogram bins. - `convert.tempo_to_lag` — Convert BPM to lag (in frames) - `convert.time_to_frames` — Convert time (seconds) to frame indices - `convert.time_to_samples` — Convert time (seconds) to sample indices - `convert.times_like` — Return an array of time values to match the time axis from a feature matrix - `convert.tuning_to_A4` — Convert tuning deviation to A4 reference frequency - `convert.z_weighting` — Z-weighting (flat/no weighting) for frequency analysis ### Music notation - `notation.fifths_to_note` — Calculate the note name for a given number of perfect fifths - `notation.hz_to_fjs` — Convert one or more frequencies (in Hz) to Functional Just System (FJS) notation - `notation.hz_to_svara_c` — Convert frequencies (in Hz) to Carnatic svara notation within a melakarta raga - `notation.hz_to_svara_h` — Convert frequencies (in Hz) to Hindustani svara notation - `notation.interval_to_fjs` — Convert an interval to Functional Just System (FJS) notation - `notation.key_to_degrees` — Construct the diatonic scale degrees for a given key - `notation.key_to_notes` — List all 12 chromatic note names as spelled according to a given key. - `notation.list_mela` — List melakarta ragas by name and index - `notation.list_thaat` — List supported thaats by name - `notation.mela_to_degrees` — Construct the svara indices (degrees) for a given melakarta raga - `notation.mela_to_svara` — Spell the Carnatic svara names for a given melakarta raga - `notation.midi_to_svara_c` — Convert MIDI numbers to Carnatic svara within a melakarta raga - `notation.midi_to_svara_h` — Convert MIDI numbers to Hindustani svara - `notation.note_to_svara_c` — Convert western note names to Carnatic svara within a melakarta raga - `notation.note_to_svara_h` — Convert western note names to Hindustani svara - `notation.thaat_to_degrees` — Construct the svara indices (degrees) for a given thaat ### Intervals - `intervals.compareTuningSystems` — Compare different tuning systems - `intervals.generateFrequencies` — Quick frequency generation utility - `intervals.interval_frequencies` — Construct interval frequencies (convenience wrapper) - `intervals.plimit_intervals` — Construct p-limit intervals (convenience wrapper) - `intervals.pythagorean_intervals` — Construct Pythagorean intervals (convenience wrapper) ### Linear algebra - `linalg.eigh` — Symmetric eigendecomposition via cyclic Jacobi rotations. - `linalg.laplacian` — Normalized graph Laplacian of a dense weight matrix. ### Clustering - `cluster.kmeans` — K-means clustering — Lloyd's algorithm with greedy k-means++ seeding. ### Display & visualization - `analyzeWaveform` — Calculates waveform statistics for analysis - `cmap` — Get a default colormap from the given data - `createInteractiveRenderer` — Creates an interactive waveform renderer with events - `drawWaveform` — Draw waveform visualization - `getStereoWaveformPeaks` — Extracts stereo waveform data for left and right channels - `getTimebasedWaveform` — Generates time-based waveform data with precise time stamps - `getWaveformPeaks` — Extracts waveform peaks suitable for visualization - `getWaveformRange` — Generates waveform data for a specific time range - `harmonic_product_spectrum` — Compute harmonic product spectrum (HPS) for pitch detection - `renderStaticSpectrum` — Renders static spectrum analysis of audio buffer - `renderStereoWaveform` — Renders stereo waveform with separate channels - `renderWaveform` — Renders waveform data to a canvas element - `specshow` — Display a spectrogram/chromagram/CQT/etc on a Canvas element - `waveshow` — Visualize a waveform in the time domain on a Canvas element ### Audio I/O — synthesis & codecs - `audioio.autocorrelate` — Autocorrelation of a signal up to a maximum lag. - `audioio.chirp` — Synthesize a linear or exponential frequency sweep (chirp). - `audioio.clicks` — Synthesize a click track at the given times or frames. - `audioio.getDuration` — Compute a signal's duration in seconds from its length and sample rate. - `audioio.getSamplerate` — Read the sample rate of an AudioBuffer. - `audioio.load` — Fetch and decode an audio file, with optional mono downmix, resampling, offset, and duration. - `audioio.lpc` — Burg LPC (real‑valued) — returns LPC denominator polynomial a[0..p], a[0] == 1 - `audioio.muCompress` — mu-law compress a signal, optionally quantizing to integer codewords. - `audioio.muExpand` — mu-law expand (decode) a companded signal back to linear amplitude. - `audioio.play` — Play the currently loaded audio through the Web Audio API, optionally looping. - `audioio.resample` — Linearly resample a signal from one sample rate to another. - `audioio.stop` — Stop the current Web Audio playback and release the source node. - `audioio.toMono` — Downmix multi-channel audio to a single mono channel by averaging. - `audioio.tone` — Synthesize a pure sinusoidal tone at a given frequency. - `audioio.zeroCrossings` — Mark the sample positions where the signal changes sign. ### Audio I/O — files & cache - `file.cache` — Get cache management interface - `file.createAudioContext` — Create a new Web Audio API context with proper configuration - `file.createVisualization` — Create audio visualization data - `file.example` — Load example audio file from remote source - `file.exampleAudio` — Get audio data as Float32Array from AudioBuffer - `file.exampleBuffer` — Load and decode audio example to AudioBuffer - `file.exampleInfo` — Get metadata for a specific example - `file.isWebAudioSupported` — Utility function to check if Web Audio API is available - `file.listExamples` — List all available audio examples - `file.saveAudio` — Save audio data as downloadable file ### Audio I/O — streaming - `fileio.cite` — Get citation information for the pleco-xa library - `fileio.createMediaStreamProcessor` — Create a real-time audio stream processor for live input - `fileio.find_files` — Get a sorted list of audio files using File System Access API - `fileio.stream` — Chunked audio reader (not true streaming). ### Audio I/O & playback - `applyLiveDoubleSpeed` — Raise live playback to double speed in real time (crossfaded playback rate, or pitch-preserving resample). - `applyLiveHalfSpeed` — Drop live playback to half speed in real time (crossfaded playback rate, or pitch-preserving resample). - `createAudioBlob` — Encode an AudioBuffer's first channel as a WAV Blob. - `decodeWav` — Decode a WAV file into planar Float32Array channels (PCM 16/24/32-bit integer and 32-bit float). - `encodeWav` — Encode planar channel data as an interleaved 16-bit PCM WAV file. - `exportBufferAsWav` — Export an AudioBuffer as a .wav file. - `findAudioStart` — Find where audible audio begins (first sample above a threshold, snapped to the nearest zero crossing). - `initAudioProcessor` — Initialize the audio processor - `loadAudioFile` — Load audio file (from URL or File object) - `loadFile` — Load local audio file from user input - `mel_to_audio` — Invert a mel power spectrogram to audio using Griffin-Lim - `mfcc_to_audio` — Convert Mel-frequency cepstral coefficients to a time-domain audio signal - `resetLiveSpeed` — Reset live playback back to normal (1x) speed. - `valid_audio` — Determine whether a variable contains valid audio data ### Playback - `playback.closeGapLeft` — Close a detected gap by shifting the audio after it left. The normalized loop end is preserved. - `playback.closeGapRight` — Close a detected gap by removing it and rescaling the loop end to the shorter buffer. - `playback.createBufferLike` — Default pure buffer factory: an AudioBuffer-shaped object backed by Float32Array channels. - `playback.detectGap` — Detect a gap (silence across all channels) after the loop end. - `playback.doubleSpeedQuantzLoop` — Double speed quantz — gapless: compress the loop content at 2x speed into half the space and re-fill. - `playback.doubleSpeedUnquantzLoop` — Double speed unquantz: compress the loop content at 2x speed in place (track length unchanged). - `playback.halfSpeedLoop` — Half speed (time stretch) a loop section. The loop region is stretched to 2x its length. - `playback.halfSpeedQuantzLoop` — Half speed quantz: time-stretch the loop content at half speed but mask it to the original length. - `playback.revealFirstHalf` — Reveal the first half of a half-speed-quantz'd loop (counterpart of revealHiddenHalf; toggles back). - `playback.revealHiddenHalf` — Reveal the "hidden" second half of a half-speed-quantz'd loop by replacing the loop window with it. - `playback.reverseSection` — Reverse a sample range of a buffer without mutating the input (copy-then-reverse). ### Experimental & creative play - `applyOperationEnhanced` — Apply a loop operation with live responsiveness — large reverse operations run in chunks with progress callbacks. - `applyQuantumOp` — Apply a single named operation (half, double, move, reverse, reset, stutter, phase, fractal) to a buffer and loop. - `buildQuantumOpList` — Build an operation list by warping a random seed through vector space and injecting preset accent bars. - `buildQuantumSequence` — Turn an operation list into a list of executable playback steps, each applying one operation to the buffer and loop. - `checkBufferSafety` — Validate a buffer and loop range, reporting whether the operation is safe along with any issues and loop metrics. - `doubleLoop` — Double a loop descriptor's length by extending its end, clamped to the buffer length. - `executeOperation` — Apply a single named loop operation (half, double, move, reverse, reset, stutter, fractal, phase) to a buffer and loop. - `generateChaotic` — Generate a chaotic sequence of loop operations driven by a logistic-map iterator. - `generateFibonacci` — Generate a Fibonacci-patterned sequence of loop operations. - `generatePrimeRhythm` — Generate a prime-number-driven rhythmic sequence of loop operations. - `generateWaveform` — Generate a sine-wave-modulated sequence of loop operations. - `glitchBurst` — Run a time-boxed burst of randomized loop glitches on an internal clock; returns a stop function. - `halfLoop` — Halve a loop descriptor, keeping the start and moving the end to the midpoint. - `isLargeOperation` — Decide whether a loop operation counts as large (long loop, high buffer share, or long file) and warrants chunked processing. - `moveForward` — Advance a loop descriptor forward by a number of samples, clamped so it never runs past the buffer end. - `playQuantumOps` — Play a quantum operation sequence with adaptive, oscillating per-operation timing. - `randomLocal` — Apply a short random burst of loop operations to the current loop and return the result. - `randomPreset` — Return a randomly chosen preset beat pattern (an eight-step operation bar) for injection into a sequence. - `randomSequence` — Build a randomized sequence of loop-manipulation steps (move, half, double, reverse, reset) for live playback. - `resetLoop` — Reset a loop descriptor to span the entire buffer. - `reverseBufferSection` — Reverse a sample range of a buffer in place (mutates the buffer). - `signatureDemo` — Build the library's fixed signature demo — narrow, move, reverse, grow, and finish — as a list of playable steps. - `startBeatGlitch` — Start a beat-synchronized glitch that fires random loop operations once per detected bar; returns a stop function. --- ## Links - Docs, guides, live demo gallery: https://plecoxa.com - Categorized API reference: https://plecoxa.com/api-by-category/ - This package ships the guides in `docs/` — no fetch needed. - Repository: https://github.com/pleco-xa/pleco-xa Generated by tools/generate-llms.mjs — do not edit outputs by hand.