Non-destructive stem processing in the browser

Summary

The whole core workflow runs locally: decoding, hit detection, timing, dynamics, transient and stereo processing, loudness measurement, synchronized A/B playback, and WAV export. Originals are never modified. Each render is rebuilt from the source using the stem's parameters and a variation number (seed), so the same settings always produce the same file. Output length equals input length to the sample.

A backend becomes necessary for accounts, syncing projects between desktop and mobile, processing while the tab is closed, and separating stems from a full mix.

Signal path per stem

Feature status

In prototype: working in CM Humanizer today. Browser: feasible locally, not built yet. Backend: needs a server. Not in browser: can't be done from a web page without a native helper.

Limits to plan for

Preset comparison

The Advanced panel has a humanization preset combobox for the selected stem. Every instrument gets Original / Bypass, its three presets and Custom. Drums also get six experimental presets (Dynamics Only, Transients Only, Stereo Only, Timing & Groove Only, Max Change, Organic Drummer). They are listening experiments, not validated settings, and are defined in one table in humanizer-engine.js (EXPERIMENTAL), next to the existing presets. Picking one writes its values into the stem's parameters, keeps the stem's variation seed, and re-renders through the existing worker path. Changing any value afterwards marks the stem Custom without discarding the edit.

Original / Bypass is a flag, not a set of neutral values. process() returns a copy of the input placed on the project timeline (same offset and length as every other render) and skips onset detection, DSP and the safety gain, so the reference cannot be altered by nominally neutral processing.

Render & Download WAV encodes the selected stem with its current settings. Export All Preset Variations renders each preset available to the stem from the original with the same seed, offset and timeline, runs the usual export checks on each file, and packs them with a manifest.json (preset, parameters, seed, source file, sample rate, format) into one ZIP. Both use the existing renderEncode/encode operations; there is no second processing path.

AI Forensics Lab

A separate page (AI Forensics Lab.dc.html) that measures recordings and compares originals with humanized versions. It is research tooling: it describes signals and does not decide how a recording was made. It shares the engine and worker with the humanizer but changes no preset or processing code.

• forensics-engine.js holds the measurement code, the metric catalogue (label, unit, algorithm), report and CSV writers, the classifier registry, dataset splitting and evaluation, and synthetic fixtures. It imports parseWav, detectOnsets, lufs (BS.1770-4), gridInfo, align, hashAudio and peaks from humanizer-engine.js, so none of those are duplicated.
• Worker reuse. humanizer-engine.js has an OPS table that runOp consults after its own operations. forensics-engine.js registers fx.analyzeWav, fx.align, fx.fixture and fx.classify there, and humanizer-worker.js imports it. The same DSP client, progress reporting, cancellation and main-thread fallback apply. Other engine changes are additive: gridInfo is exported, and mixEnd can return the mixed audio (keep: true) for the hand-off.
• Memory. Imported files stay as File objects (disk-backed). One file at a time is read, hashed (SHA-256), transferred to the worker, decoded and analyzed; only the report returns: summary values, 0.5 s series, a 72-band spectrogram thumbnail and 1200 peak columns. Audio is decoded for playback only for the current A and B.
• Whole files. Every sample is read; STFT frames run to the last sample and the report states the frame count and last sample covered. Alignment uses the engine’s cross-correlation over the first 90 s (±1 s search), then sample-level refinement.
• Blind analysis. Analysis inputs are bytes, file name and SHA-256. Provenance can only be entered after a recording is analyzed and is stored in its own IndexedDB store, keyed by SHA-256. Dataset entries (category, lineage, split) are another store. Reports never contain provenance; exports list it under a separate key.
• Classifiers. None are installed, so every surface shows “Classifier unavailable”. registerClassifier() rejects descriptors without name, version, licence, source, threshold, input seconds, sample rate and limitations. No heuristic score exists anywhere in the code. Candidates (SONICS SpecTTTra, AST-based heads, spectral fakeprint models) are listed with what still needs review.
• Dataset. Splits come from a hash of the normalised lineage (60/20/20), so all versions of a source recording share a split. evaluate() computes confusion matrix, precision, recall, F1 and FPR for a trained model’s scores on the test split.
• Hand-off. “Analyze in AI Forensics Lab” in a project encodes copies (32-bit float) of the original stems, humanized stems, original stem mix and humanized mix into an IndexedDB inbox with preset, parameters and seed, and notifies an open Lab over BroadcastChannel.

Known limits: WAV only (no MP3/FLAC decoding in the analysis path, by design, so measurements are not affected by a browser decoder); sample peak, not true peak; the periodicity tempo can be off by an octave; alignment cannot correct drift or different sample rates; playback may resample to the device rate; the in-page alignment estimate uses one 3 s window, so between versions with per-hit timing changes it reports that window’s local mean shift (a few samples), not a global offset. Lab tests: Forensics Tests.

WAV encoding

encodeWav() in humanizer-engine.js writes a canonical 44-byte RIFF/WAVE header (fmt chunk size 16, then data) for three formats and rejects any other bit depth with an error:

• 16-bit PCM (format tag 1). Samples are clipped to ±1 and scaled by 32767 with round-to-nearest, so +1 → 32767 and −1 → −32767. No dither. Before this fix a 16-bit request produced a malformed file; the app’s export menu never offered 16-bit, so only test fixtures were affected.
• 24-bit PCM (format tag 1). Clipped to ±1, scaled by 8388607, little-endian three-byte samples. Unchanged by this fix.
• 32-bit float (format tag 3). IEEE float written as-is, no clipping, so values above full scale survive. Unchanged. The export verifier (scanWav) used to clip its reference to ±1 for every format, which would have failed a float export holding samples above full scale; it now clips the reference only for PCM.

Regression test X3b encodes each format at 1, 2 and 6 channels (44.1, 48 and 96 kHz, including an odd sample count) and checks every header field (chunk IDs, RIFF and data sizes, format tag, channels, rate, byte rate, block align, bits), exact 16-bit codes at ±1, 0.5, ±1.5 and 0, round-trip decoding through the engine’s parser and the browser’s decoder, and a quantisation-error bound per sample (float must be bit-exact). It also runs a 16-bit export through the worker’s encode-and-verify path and checks that an 8-bit request is rejected. The app exports 24-bit PCM or 32-bit float; 16-bit is available to the engine and tests.

Persistence, backups and cloud sync

Offline-first: IndexedDB in the browser is the primary store and the app works fully without Supabase, without signing in and without a network. Cloud sync is an optional copy of metadata in a dedicated CM Humanizer Supabase project. A local save never waits for, or depends on, the cloud.

Local schema and migrations

One database, shared with AI Forensics Lab so there is no second store for the same data. It keeps the Lab's original name (cm-forensics). Migrations are forward-only and run in order from the stored version; no store is cleared.

• v1 (existing, Lab): inbox, dataset (lineage, category, train/calibration/test split), provenance (keyed by SHA-256).
• v2 (added): projects (UUID, name, tempo, meter, status, timestamps, schema version, stems with source, active version, offset, preset, parameters, seed, output trim, mute/solo, source SHA-256 and format; export configuration; rev / syncedRev / serverRev / conflict), sources (original audio Blob, one record per SHA-256 of the file bytes), reports (compact Lab measurement reports), renders (disposable cache), outbox (sync operations), meta (settings, cloud configuration, sync state, schema record).
• Decoded Float32 audio and renders are not persisted. Reopening a project decodes the stored originals and re-renders from the saved settings and seed, which the engine reproduces exactly.
• Autosave is debounced (800 ms; slider drags write once). The local revision increases only when the content changes. Save writes immediately. Quota and other storage errors show "Storage error" and leave the last saved record intact, because each write is a single transaction.
• Projects created before this version existed only in memory and cannot be recovered. The demo projects are generated on open; "Save as project" stores a copy.

Backups

A store-only ZIP with manifest.json (format cm-humanizer-backup, version 2) and, for complete backups, audio/<sha256>.<ext>. Import checks every archive path against that layout (no "..", absolute or unexpected names), rejects compressed, encrypted or oversized entries, verifies CRC-32, declared size and SHA-256 of each file, validates the manifest and migrates older versions explicitly (v1 → v2). An identical project is reported as already present; a different project with the same id is imported as a copy. Nothing is overwritten. A metadata-only backup opens with "Project metadata restored. Source audio is not available on this device." and a Relink Source Audio action that accepts only files with the recorded SHA-256.

Supabase tables and ownership

projects → project_stems (instrument slot) → stem_versions (file name, provider, SHA-256, format, offset, active) → processing_configs (preset, JSONB parameters and output settings, validated and versioned). forensic_reports, recording_provenance and research_dataset_entries hold research records; sync_metadata records applied operation ids. UUID keys throughout; owner_id references auth.users. RLS is enabled and forced on every table with separate SELECT/INSERT/UPDATE/DELETE policies on auth.uid(); child tables check ownership through the parent project. anon has no privileges. No audio is stored in PostgreSQL.

Sync protocol and conflicts

• Only after sign-in (email code or magic link) and an explicit "Enable cloud sync". With sync off, nothing is queued or sent.
• Each save queues an operation with a stable UUID in the outbox; unsent operations for the same project are coalesced. Sent operations keep their id, and the server stores the result per id, so a retry after a lost response is applied once.
• cm_push_project applies a snapshot only if the server revision equals the client's base revision (optimistic concurrency, not timestamps). A mismatch returns the remote snapshot: the project is marked Conflict, local edits stay in place, and the user chooses this device's version, the cloud version, or both (this device's becomes a new project).
• Deletion is a soft delete with a tombstone on both sides. A push to a deleted project is answered "deleted", a database trigger refuses to clear deleted_at, and a pulled deletion is applied locally only when there are no unsynced edits (otherwise it is a conflict).
• Offline or 5xx/429 failures retry with exponential backoff (1 s doubling, ±50 % jitter, capped at 60 s). An expired session shows "Sign-in expired" and keeps the queue. "Cloud synced" appears only when the server has acknowledged the project's current local revision.
• Validation before queuing. Every project is checked against the database constraints before it is queued (tempo empty or 20–400 BPM, beats per bar 1–16 as a whole number, name length, instrument types, SHA-256 format, output-trim ranges, snapshot size). The tempo and beats-per-bar fields refuse other values with an inline error and keep the saved value; New project cannot be created with them.
• Per-project isolation. Queued projects are pushed one at a time. One that fails the local check (for example a record saved by an older version with 999 BPM) or that the server refuses (constraint or permission error) stays queued with its reason, is shown as "Sync blocked" with the reason in its header, and is retried after its next change. The rest of the queue continues. Only network failures (retried with backoff) and an expired sign-in pause the whole queue.
• Pagination. cm_pull_projects(p_since, p_since_id, p_limit) returns rows after an (updated_at, id) cursor in that order, at most 1000 per page (the client asks for 500). The id breaks ties between rows written with the same timestamp. The client pages until a short page and saves the cursor after each page, so an interrupted pull resumes. Each sync starts 10 s before the stored cursor, so a row whose transaction committed late with a slightly older timestamp is still returned; rows read twice are ignored by revision. Deletions are rows with deleted_at set.
• One account per browser profile. The first account that syncs on a device is bound to its local data; a different account is refused before any request. There is no secure in-app account reset yet, so a second account needs a separate browser profile.

Audio storage policy

Audio stays local. Sync carries hashes and format facts only. supabase/optional/audio_storage_bucket.sql sketches a private, per-user Storage bucket for a future opt-in audio backup; it is not applied and the client has no upload code. To move audio between computers, use a complete backup.

Verification status: local persistence, backups, validation and the sync client are covered by Persistence Tests.dc.html (22 tests; the sync tests use an in-memory mock of the RPCs and RLS, not a real backend). The four SQL migrations, supabase/tests/rls_isolation.test.sql (22 pgTAP checks), email sign-in and two-device sync have not been run against a Supabase project yet, so cloud sync is not validated; see SUPABASE_SETUP.md.

Remaining limitations: no in-app account switch (use a separate browser profile); a pull can miss a row only if its transaction commits more than 10 s after taking its timestamp; audio is never synced, so a second device needs a complete backup or relinking; blocked projects retry only after their next edit or Sync now with a change.

Output trim and peak headroom

Humanization can raise the crest factor considerably: on the LANDR drum stem from Seven Against Four, Organic Drummer took the crest factor from 18.6 to 25.6 dB, and the existing safety stage left the peak at −0.01 dBFS. The Output section in the Advanced panel restores headroom with a constant gain that sits outside the humanization DSP.

1. Original audio, placed on the project timeline.
2. Existing humanization DSP (process(), including its existing safety gain to 0.999). Unchanged.
3. Manual output trim: −24 to +12 dB in 0.1 dB steps, default 0.
4. Optional automatic peak trim: if the sample peak after manual trim is above the target, one further constant reduction brings it to the target. It never raises the level.
5. Output validation: header, sample count, rate, channels, silence profile and a sample-by-sample comparison with the trimmed render.
6. WAV export.

• Gain architecture. outputPlan(peak, settings) in humanizer-engine.js computes manual, automatic and total gain from the render's measured sample peak; applyOutput() multiplies every sample by the total linear gain. No limiter, compressor, saturation or clipping is involved, and timing and sample count cannot change. A total gain of exactly 1 returns the input untouched.
• Re-render avoidance. Output settings are stored on each stem version (stem.output), outside params, so they are not part of the render key. Changing them never re-runs onset detection or timing, never changes the variation seed, never switches the preset to Custom, and preset changes keep them. The worker's process operation returns the render's sample peak and integrated loudness, so the summary updates instantly.
• Sample peak vs. true peak. All peak values and the target are sample peaks in dBFS: the largest absolute sample value. A true-peak meter (dBTP, ITU-R BS.1770 4× oversampling) can read higher, because the reconstructed waveform can exceed full scale between samples. A −1 dBFS sample-peak target therefore does not guarantee −1 dBTP.
• Loudness. Loudness before trim is measured on the render (BS.1770 integrated). Loudness after trim is calculated as before + total gain, which is exact for a constant gain because the gating is relative; the panel and report label it as calculated.
• A/B loudness matching. Playback applies the same output gain to B through its gain node, and the mix loudness for B is measured with output gain included, so "B … LUFS exported" is the level of the exported files. Loudness match then adds a separate playback-only gain to B, shown as the listening level. It never reaches an export.
• Integer PCM and float. 32-bit float export keeps samples above 0 dBFS exactly when automatic peak trim is off (verified bit-exact). 24-bit and 16-bit PCM keep the encoder's existing behaviour: samples are clipped to ±1 before quantisation. The panel warns before export when the trimmed peak is above 0 dBFS at 24-bit, and the export reports the number of clipped samples.
• Bypass. With 0 dB trim (or automatic trim that has nothing to reduce), Original / Bypass stays bit-transparent. Any gain on a bypassed stem is labelled "Bypass + Output Trim" in the panel, file names and reports.
• Non-destructive. The original buffer and the humanized render are never modified; output gain is applied to a copy in the worker at export and as a playback gain node in the browser. Reports (processing report v3, preset-variation manifest, Lab hand-off metadata) record manual, automatic and total gain, the sample-peak target, and peak and loudness before and after. Output gain is a level change and is not treated as a change in humanization.

Alignment estimate vs. global timing shift

align() finds a coarse lag from energy envelopes, then refines it with a sample-level cross-correlation on the single loudest 3 s window. That is right for versions that differ by a fixed delay. With per-hit timing humanization each hit moves on its own, so the refined lag is the average shift of the few hits in that window, not the shift of the whole file. Test G2 previously required |align()| ≤ 2 samples and failed at −5 samples (Drums · Loose, seed 777). Investigation: the mean applied shift over all 62 hits is +0.030 ms (+1.3 samples); a whole-file cross-correlation gives +3 samples; timing-only renders give the same −5, and dynamics- or transient-only renders give 0. So the −5 comes from the timing of hits inside the refinement window, and there is no global shift. G2 now checks the mean applied shift (≤ 2 samples) and the whole-file lag against its statistical resolution (2 × RMS shift / √hits, here ±5.9 samples), and reports align()’s value for information. Tolerances in other tests and the processing code are unchanged.

REAPER handoff

Exports are WAV at the source sample rate, either 24-bit PCM or 32-bit float, always stereo (mono sources become dual-mono, or widened if width is above 100%). File names start with a track number so REAPER imports them in order. Drag all files to the start of the arrange view and choose one track per file. A generated .RPP project with named tracks, colours and the tempo already set is the next step; it is plain text and can be written in the browser.

Suggested build order

1. Move processing into a Web Worker with transferable buffers, so the UI stays responsive on 5-minute stems.
2. Store sources and settings in OPFS/IndexedDB so projects survive a reload.
3. Replace the slice-and-crossfade timing with WSOLA for sustained material (keys, pads, clean guitars).
4. Exact BS.1770 filter coefficients and a true-peak meter.
5. .RPP export, then optional cloud sync for mobile review.