Song Structure & Hierarchical Harmony — Research Report

00Summary

The hierarchy idea is well supported by how published chord and key systems work (transition-penalised decoding, whole-piece context for local key) but is not yet proven on our data (ESTIMATE); most of it can be built on CPU from pieces we already run. The biggest wins are not a higher whole-song key score. They are cleaner chord charts (bar-level decoding, no flicker), real section boundaries instead of four equal slices, and per-section keys and modulation flags that we can trust.

~0.70MEASURED strict boundary hit-rate F (±0.5 s, HR.5F .703) on SongFormBench-Harmonix — SongFormer (HX), their paper/README

~0.78MEASURED HR3F (±3 s) .784, same benchmark and model

~0.81MEASURED frame-level label ACC .807 on SongFormBench-Harmonix (7 label classes; a different SongFormer training variant, HX+E+H+G)

84%MEASURED ChordFormer MajMinor WCSR 84.09, Sevenths 72.28 — 1,217-song Isophonics+Billboard+MARL set, 5-fold CV

Recommendations

Start with CPU-only wins (phases 0–2): a benchmark harness, a bar-level chord Viterbi on the chroma we already compute, and self-similarity sections snapped to SANCT bar lines. No new licences, no GPU.
Treat deep section models as a second opinion, behind a licence gate (phase 3). allin1 is MIT code; SongFormer is stronger on paper but depends on MuQ, whose published weights are CC-BY-NC. That is a commercial-use blocker until cleared.
SANCT stays sacred. Everything here reads beat_times/downbeat_times and never rewrites them. Section edges are snapped to the grid, not the other way round.
Hierarchy is not “shorter windows”. The plan adds priors and pooling over bar-level states; it does not shrink the analysis window, which would make chroma noisier. See 4b.
Measure first. The repo has no key or downbeat ground-truth benchmark today (section 1). Phase 0 fixes that; without it every gain below is an estimate.

How to read the numbers

Accuracy figures are from papers that report them on specific datasets. They do not transfer 1:1 to our users’ uploads (stems from Demucs, loop-based beats, mixed genres). Treat them as ceilings, not promises. All projected gains (roadmap, compute adders, “expected” anything) are ESTIMATE. Anything I could not fetch today is tagged unverified and listed at the bottom. A claim-by-claim ledger with the sources re-checked is in Claim ledger.

01How our key detection works today

Read from the repo (read-only). Code lives in afcut_engine/harmonic.py; webapp/app.py:_detect_key is now a thin wrapper that delegates to it. verified in repo

Stage	What it does	Where
Input	Full mix (the pack’s mixed audio). No stems are used for key. Mono downmix, resampled to 22.05 kHz, capped at the first 120 s.	harmonic.py ANALYSIS_SR, MAX_ANALYSIS_SECONDS; app.py:7543, :28930
Harmonic isolation	HPSS (STFT n_fft 1024, kernel 17, margin 1/3) keeps the harmonic part, so drums do not smear the chroma.	_harmonic_component
Chroma	CQT chroma on the harmonic part. Frames are RMS-weighted and gated at 5% of the 95th-percentile energy, then averaged to one 12-bin vector.	_harmonic_chroma, _weighted_chroma
Profiles	Krumhansl-Kessler major/minor only. No Temperley, edma/edmm or learned profiles.	KRUMHANSL_MAJOR / MINOR
Scoring	Rank (Spearman-style) correlation against all 24 rotated profiles. Rank transform makes it robust to the overall loudness/shape of the chroma vector.	_rank_keys
Confidence	Margin between best and second-best, `margin / 0.125`, deliberately no floor. Returns the top 3 alternatives. Code comments say it was tuned on a small calibration set and that a wrong answer still reached 0.39 — i.e. confidence is a hint, not a probability.	_calibrate_confidence, _alternatives
Cross-checks	CENS chroma on an 11 kHz / 45 s excerpt gives an `key_agreement` flag; a triad-pair “rival tonality” test (threshold 0.16) flags relative/neighbour confusions; a guard returns “indeterminate” for silence/DC/degenerate scores.	detect_key_advanced, _rival_tonality_gain, _degenerate_key_reason
Per-section key	Not structural. `detect_per_section_keys` splits the audio into 4 equal-length slices and runs the same scorer on each.	harmonic.py:734
Chords	`chord_timeline`: for each beat window (from the SANCT grid, read-only) take triad-template cosine argmax over 12 roots × a few qualities. Merge identical neighbours, cap at 200. No HMM, no transition cost, no bar awareness, no 7ths. Confidence from template margin.	harmonic.py:1099
Where it runs	`_harmonic_layer` on the worker cache-miss path only (never on the API origin). Measured ~13.6 s per ~4 min song, ~0.9–1.1 GB peak on the cx33 worker box.	app.py:~32335; docs/reports/2026-10-09-song-intelligence-strategy.md
Section prototype	Offline-only `tools/song_intel_demo.py`: per-bar energy (RMS + onset density + brightness), novelty peaks with a 4-bar minimum, labels from energy rank (Intro/Verse/Build/Drop…). Not wired into the product.	tools/song_intel_demo.py

Why it is accurate (and where that is a claim, not a measurement)

It reads harmonic content only (HPSS), uses CQT chroma and a rank-based comparison — all robust choices that avoid the classic failure of full-mix chroma polluted by drums.
It reports alternatives and a margin, so relative-key confusion (e.g. “D# minor, could also be F# major” in the 2026-10-09 demo) is surfaced instead of hidden.
It is an honest baseline, not state of the art: one profile family and a single global vector. Literature suggests profile choice alone matters by genre (section 4).

Data we have to benchmark it

Essentially none. No tests/ directory, no ground-truth key file, no evidence folder with labelled keys. The code comments mention two published-key demo tracks used for calibration (one still came out wrong: a C♯ minor piece detected as D minor, per scout notes unverified: not re-checked). mir_eval==0.8.2 is already pinned in requirements.txt, so scoring tools are free. The founder’s “very accurate” is a plausible but unmeasured claim — phase 0 of the roadmap turns it into a number.

Also noted: the Track 12 plan (docs/plans/2026-09-14-track12-bpmandkey-harmonic-analysis.md) still lists chord_timeline as to-be-built and cites _detect_key at app.py:4772; both are stale (it now lives at app.py:5741, and chords exist). Not edited here.

02Section / structure segmentation

Libraries and models

System	What	Reported accuracy	Licence	Notes
allin1 (All-In-One)	Joint beats, downbeats, section boundaries and labels (intro/verse/chorus/bridge/outro…). Takes demixed stems. WASPAA 2023.	MEASURED Harmonix, 8-fold CV (their paper): beat F1 .958, downbeat F1 .915, boundary HR.5F .660, label pairwise-F .738 (on SongFormBench-Harmonix, measured by the SongFormer authors: HR.5F .596, HR3F .730, ACC .740)	Code MIT verified (GitHub licence API). Weights licence not stated in the README unverified. Needs NATTEN and madmom (from git).	Speed per README: 33 min of audio in 73 s on RTX 4090 + 14-core CPU. Trained on Harmonix only (pop/electronic skew).
SongFormer (Oct 2025)	Fuses MuQ + MusicFM representations; trained on 14k+ songs (SongFormDB) with mixed label schemas. arXiv 2510.02797.	MEASURED SongFormBench-Harmonix (200 Harmonix + 100 Chinese-pop songs for the full bench): HR.5F .703, HR3F .784 (HX-trained variant); ACC .807 (HX+E+H+G variant) — best figures across the paper’s variants, not one model. RWC-Pop (HX+E+H+G variant): HR.5F .650, HR3F .800, ACC .812	Code CC-BY-4.0 (README “License” section) verified. The README tells users to fetch MuQ-related weights; MuQ’s README states its published weights (MuQ-large-msd-iter, MuQ-MuLan-large) are CC-BY-NC 4.0 verified. We did not establish which exact checkpoint SongFormer loads or its own weights licence, so the NC conflict is likely, not confirmed commercial blocker until cleared	README: 2–4 s per song on an L40 GPU (excl. model load). Beats Gemini 2.5 Pro on strict boundaries (HR.5F .423) but Gemini is slightly better at ±3 s (.813).
MSAF	Classical framework: many boundary/label algorithms (novelty, spectral clustering, etc.).	Older algorithms; no 2024–26 numbers fetched unverified	MIT	Useful as a reference/ablation, not as the main engine.
librosa	Recurrence matrix, self-similarity, agglomerative clustering (`librosa.segment`). We already ship it.	Boundary quality depends on features; no benchmark of our own	ISC unverified	Cheap CPU baseline. Our building block for SSM-on-bar-grid.
MusicFM / MuQ / MERT	Self-supervised music encoders; the strongest structure models fine-tune on top of them (SongFormer, and the “MusicFM-Zhang” and “MuQiter” baselines in its table).	MusicFM-Zhang: ACC .725, HR.5F .640; MuQiter ACC .772 (cited in SongFormer, Harmonix subset)	MuQ code MIT, weights CC-BY-NC 4.0; MusicFM: repo LICENSE file says dual MIT / Apache-2.0 (GitHub shows NOASSERTION) verified; weights licence not checked unverified	Encoders are heavy; need a GPU in practice.
LinkSeg	Recent supervised structure model (baseline in SongFormer paper).	ACC .780, HR.5F .630, HR3F .762 (Harmonix subset)	unverified	Not evaluated by us.

Realistic accuracy and the honest ceiling

What the numbers mean

On the best public benchmark, a strict ±0.5 s boundary F-measure tops out near 0.70; relaxed ±3 s is near 0.78–0.80. An F of ~0.70 means a substantial share of strict boundaries are still missed or spurious even for the 2025 state of the art (F mixes precision and recall, so “one in three” is only a rough reading). MEASURED SongFormBench-Harmonix, HR.5F/HR3F.
Frame-level label accuracy ~0.80 MEASURED (ACC, SongFormBench-Harmonix) is with a seven-class label set. It drops as labels get finer.
Snapping predicted boundaries to the nearest bar line helps strict F, because annotators mark bar starts (estimate — to be proven in phase 0).

By genre — what we can and cannot say

The benchmarks are Harmonix (mostly pop/rock/electronic, unverified composition), RWC-Pop (J-pop) and 100 Chinese pop songs. No fetched number covers hip-hop, loop-based electronic, jazz or classical.
Our own 2026-10-09 prototype note: sections were “believable” on a beat-driven track and wrong (“Verse”) on orchestral music.
estimate Loop-based music (hip-hop, house) has structure carried by layers entering and leaving, not by harmony or melody changing. Chroma self-similarity is weak there; energy/stem-activity features matter more. Expect lower label accuracy and boundaries that are real but ambiguous (drop vs. build).
estimate Pop/rock with verse-chorus form and vocals is the best case.

The human ceiling is also below 1.0: annotators disagree on where sections start and what to call them (see risks). I could not fetch a sourced inter-annotator figure today unverified.

Commercial and competitor approaches

Product	What they do	Evidence
rekordbox	“Phrase analysis” marks track phrases; auto hot-cue/memory-cue placement uses it and is skipped when phrases cannot be analysed or the track is too short.	verified rekordbox 7 FAQ
Moises	“Chord Finder”: chords synced to playback, Easy/Medium/Advanced voicings, capo mode, editable, exportable chart. No accuracy figure published on the page.	verified moises.ai
Spotify audio analysis	Spotify’s 2024-11-27 notice lists Audio Analysis among endpoints removed for new apps (registered on/after that date) and for development-mode apps without a pending extension request; apps that already had access are unaffected. Not an option for us.	verified Spotify blog
Mixed In Key, Serato, Ableton, Logic, Cubase, Chordify, Songsterr, Cyanite, Musiio, AudioShake, LANDR, Beatport	Not fetched. Our own earlier note (2026-10-09) describes MIK/rekordbox/Splice features “based on established product knowledge”. Chordify’s site returned 403 to automated fetch.	unverified see unverified list

Take-away: phrase/section bands on the waveform are now table stakes in DJ software, but nobody publishes accuracy. Our differentiator is honest confidence + bar-exact edges on our own grid.

03Chord recognition & harmonic rhythm

System	Approach	Reported accuracy	Licence	Notes
Chordino / NNLS-Chroma (Mauch)	NNLS-chroma + HMM/Viterbi over a chord dictionary; Vamp plugin.	No fresh number fetched unverified	GPL-2.0 (repo) verified	Classic, interpretable, CPU. GPL means keep it a separate process or skip it.
madmom chords (deep chroma + CRF)	CNN chroma + CRF decoding.	unverified	Code BSD-style; model files CC BY-NC-SA 4.0 verified LICENSE	Non-commercial weights ⇒ off-limits for the paid product.
BTC (Park et al., ISMIR 2019)	Bi-directional Transformer; maj/min or large vocabulary.	MEASURED as a baseline in the ChordFormer table: “BTC+CNN” MajMinor 49.00 (the paper’s own re-run on its 1,217-song split; not BTC’s own published result). Training sets listed from memory unverified	Code MIT verified repo; weights/training audio status unverified	Old but usable; librosa-based features.
ChordFormer (Feb 2025)	Conformer on CQT, structured chord representation, re-weighted loss, CRF decoding to cut spurious transitions.	MEASURED 1,217-song Isophonics+Billboard+MARL set, 5-fold CV, WCSR: Root 84.7, MajMin 84.1, Sevenths 72.3, MIREX 83.6 (CNN+BLSTM baseline 83.4 / 82.6 / 69.8 / 81.5)	Weights/code release unverified	Gains over the CNN+BLSTM baseline in the same table are +1.3 (root), +1.5 (MajMin), +2.5 (7ths) points MEASURED. our reading progress is incremental.
Chord Recognition with Deep Learning (Dec 2025)	Thesis revisiting methods.	Findings: poor on rare chords; pitch augmentation helps; generative-model features do not	n/a	Confirms the plateau.
Essentia chord detection, autochord, CREMA	Template / deep options.	unverified	Essentia is AGPL-3.0 verified	AGPL is a network-copyleft risk for a hosted service.

How to stop beat-by-beat flicker

Decode on bars, not beats. Aggregate beat-synchronous chroma into per-bar vectors using the SANCT downbeats, then run Viterbi over bars. A bar is the natural unit of harmony in most pop/electronic music.
Hierarchical resolution choice. Per bar, allow three hypotheses — one chord, two half-bars, or four beats — and charge a penalty for each extra change. The finer split only wins when the evidence margin is large.
Phrase-aware prior. Changes are cheaper at bar 1 of a 4/8-bar phrase than mid-phrase; repeated phrases should decode to the same chords (use the section SSM to tie them).
Transition and self-loop costs (the HMM/CRF trick used by Chordino/madmom/ChordFormer). ChordFormer’s paper explicitly credits CRF decoding with reducing excessive transitions.

Harmonic-rhythm priors — what is and is not sourced

“Most pop is one chord per bar or per two bars” matches practice and our own demo (about 115 chords for a ~4-minute song, which is too dense), but I did not fetch a corpus study (e.g. McGill Billboard statistics, de Clercq & Temperley) unverified. Plan: measure chords-per-bar on the Billboard annotations in phase 0 and set the penalty from data, per genre bucket.

Annotation ambiguity caps all scores: even expert annotators disagree on chord labels, which ChordFormer cites as a cause of overlapping class boundaries. A sourced agreement percentage was not retrieved unverified, so the often-quoted “80–85% ceiling” is not used here.

04Local / segmental key detection

Topic	What we know	Tag
Key CNN (Korzeniowski & Widmer)	Genre-agnostic “AllConv” model, MIREX weighted score `w = r_c + 0.5 r_fifth + 0.3 r_relative + 0.2 r_parallel`. GiantSteps (electronic, Beatport previews): w 74.6 (correct 67.9, fifth 7.0, relative 8.1, parallel 4.1, other 12.9). Billboard: 85.1 (correct 79.9). Classical: 96.6. arXiv 1808.05340.	verified
Genre sensitivity	An earlier paper by the same authors finds systems tuned on pop perform subpar on electronic music, and that errors concentrate on relative and parallel keys. arXiv 1706.02921.	verified
Profile families	Essentia’s `KeyExtractor` exposes `diatonic, krumhansl, temperley, weichai, tonictriad, temperley2005, thpcp, shaath, gomez, noland, faraldo, pentatonic, edmm, edma, bgate, braw`; default bgate. Profile names such as `edma/edmm` suggest electronic-music-specific profiles; the docs page itself gives no accuracy comparison, and the claim that KK fits electronic worse rests on the Key-CNN authors’ cross-genre finding below, not on Essentia our inference. Essentia docs.	verified (names); accuracy by profile unverified
Local key / modulation	The Key-CNN paper itself notes that local-key performance on short excerpts depends on the harmonic coherence of the whole piece (1808.05340 abstract) — i.e. local windows need global context. Dedicated modulation-detection papers were not found in my search unverified. Truck-driver modulations (final chorus +1 or +2 semitones) are well known in pop unverified frequency.	mixed
Stem-aware chroma	No source fetched. Reasoning: bass stem gives a strong root estimate; “other”/harmonic stems remove drums and vocal melody-only passages. HPSS already approximates the drum removal. A test is in the ablation list.	estimate
Tuning	Our CQT chroma path uses librosa defaults; whether tuning is estimated per track was not confirmed in code unverified. Cheap to add: estimate tuning offset and re-centre before chroma.	unverified
MIREX audio key results	music-ir.org wiki pages were not reachable from this environment (connection failed). The key-CNN paper cites MIREX 2017 results.	unverified

What section structure adds to key (the core idea)

Pool across repeats. If chorus 1, 2 and 3 are the same section, sum their (energy-weighted) bar chroma before scoring. Noise from vocals/fills averages out; this is a bigger effective sample for the same key.
Per-section key from real sections, not four equal slices. A verse in the relative minor no longer drags a quarter-slice vote.
Modulation by transposed repetition. If the last chorus matches earlier choruses after rotating its chroma by k semitones (k=1,2) with high similarity, that is strong evidence of a key change, much stronger than a lone key-vote flip.
Bidirectional reconciliation. Whole-song key is the prior for each section (high self-transition in a 24-state key HMM over bars); a section overrides it only after ≥8 bars of consistent evidence.

4b · Hierarchical inference is not the same as shorter windows

These are two different ways to get “finer” harmonic output, and they pull in opposite directions. The report proposes the first and rejects the second.

Hierarchical harmonic inference (what we propose)

Keep the evidence window at a musical unit (a bar, from the SANCT grid) and decode a sequence of bar-level states with Viterbi.
Priors flow down the hierarchy: song key → section key → phrase position → bar. A change is cheap on bar 1 of a phrase and dear mid-phrase; diatonic chords are cheaper than chromatic ones; a harmonic-rhythm prior defaults to one chord per bar.
Evidence is pooled across repeats: choruses 1–3 share chroma/posteriors, so noise averages out as the effective sample grows.
A finer split (half-bar, beat) is accepted only when its likelihood gain beats a penalty. Resolution is earned, not assumed.

Just shrinking the window (what we avoid)

Cutting the analysis window (say from a bar to a beat, or a 4-slice key vote to 16 slices) gives each estimate less audio, so chroma is noisier, passing tones and vocal melody dominate, and the argmax flips.
With no transition cost or prior, each window is decided independently ⇒ flicker (this is exactly today’s chord_timeline: beat windows, argmax, no HMM, ~115 chords in a ~4-minute song).
For key, shorter slices make the vote less reliable, not more local: the Key-CNN authors note local-key classification of short segments has to consider the harmonic coherence of the whole piece.

Claim	Evidence	Label
Per-frame/window argmax over-produces chord changes; adding transition-penalised decoding fixes it	ChordFormer (arXiv 2502.11840): taking the top class per frame “often results in excessive transitions between adjacent chords, as it lacks a mechanism to impose penalties for immediate changes”; they add a linear-chain CRF for “more coherent chord transitions”. Chordino/NNLS also decodes with an HMM/Viterbi. We did not extract an isolated CRF-on/off score.	MEASURED (mechanism stated by authors) / no isolated number unverified
Local key needs whole-piece context	Korzeniowski & Widmer (arXiv 1808.05340), abstract: one must “consider the harmonic coherence of the whole piece when classifying the local key of short segments of audio”.	verified (statement), not a head-to-head
Shorter windows = noisier chroma	Signal-processing reasoning: fewer frames per estimate means higher variance, and a beat holds fewer harmonic cycles than a bar. No window-length sweep was fetched or run.	ESTIMATE (reasoned, not measured)
Pooling repeated sections improves key/chord posteriors	Standard averaging argument; no paper fetched that measures it for key. Depends on repeat clustering being right.	ESTIMATE
Harmonic-rhythm prior (about one chord per bar in pop/electronic)	Plausible and consistent with our own dense demo, but no corpus statistic was fetched.	unverified → ESTIMATE
Bar-level Viterbi beats beat-level argmax on our audio	Not measured. Falsifying test, to run in phase 0/1 (ablation 7): same chroma, same beat grid; compare (a) beat argmax, (b) shorter windows (half-beat), (c) bar Viterbi with priors. Report MajMin WCSR and chord changes per bar. The hierarchy claim fails if (c) does not beat (a) on flicker at equal or better WCSR.	ESTIMATE — hypothesis with a stated test

One caution cuts the other way: hierarchical priors can over-smooth real fast changes (jazz, passing chords) and wrong section clusters can tie unrelated bars together. That is why the sketch lets half-bar/beat hypotheses win on strong evidence and why low-confidence songs fall back to beat-level output.

05Beat / downbeat tracking as the grid

Tracker	Reported	Licence	Notes
SANCT beat_engine (ours)	No benchmark in repo. Downbeats come from a beat-phase vote over bars of 4 beats.	Ours	Sacred. librosa-based (HPSS tempo probes); no madmom/Beat This! dependency seen in `beat_engine.py`. Exposes `beat_times` and `downbeat_times`.
Beat This! (ISMIR 2024)	MEASURED GTZAN test-only set (models trained with 8-fold CV on other data): beat F1 89.1, downbeat F1 78.3 without DBN (77.4 with DBN, but better downbeat continuity: CMLt 73.3 vs 67.3). ~20 M params; the paper’s smaller model scores 88.8 / 77.2. Their ablation table on a validation split reports higher numbers (92.6 / 85.4) — do not mix the two.	MIT code and weights verified; note training data partly copyrighted	Authors admit failures on underrepresented genres and non-periodic beats; weaker continuity metrics. arXiv 2407.21658.
allin1 joint model	Harmonix: beat F1 .958, downbeat F1 .915 (different data from the row above — not comparable)	MIT code	Joint training helps all four tasks (ablation in paper).
madmom DBN	Our internal notes quote ~0.83–0.88 beat accuracy unverified source	Code BSD-style; models CC BY-NC-SA 4.0 verified	PyPI release only supports Python<3.10 and numpy<1.20, per Beat This! README verified; install from git.
Beat Transformer, SpecTNT-TCN, BEAST	MEASURED Beat Transformer: “up to 4% point improvement in downbeat tracking accuracy over the TCN architectures” (abstract; separate finding: demixed input beats non-demixed); BEAST: online, <50 ms latency, beat F1 80.04, downbeat F1 46.78	Abstracts only checked unverified beyond that	Research baselines; 2209.07140, 2312.17156.

Why downbeats matter so much here

Everything in the hierarchy — sections on bar lines, bar-level chords, 4-bar phrases — assumes the bar phase is right. If the downbeat is one beat off, every bar chroma straddles two real bars and chords smear. Public downbeat F-measures (MEASURED ~0.78 on GTZAN for Beat This!, ~0.92 on Harmonix for allin1; different datasets) say a meaningful fraction of songs will have an off-phase or wrong-meter grid. Phase 0 must measure SANCT downbeat accuracy against annotated data (read-only), and the pipeline should expose a bar-phase confidence so low-confidence songs fall back to beat-level or whole-song output.

SANCT rule for this project

New code consumes the SANCT grid; it never re-derives, snaps or reorders it. Other trackers (Beat This!, allin1) may run as diagnostics to flag grids that look wrong; they must not feed back into cut points. Section/chord boundaries are expressed as indices into the SANCT bar list, so they cannot drift from the grid by construction.

06Proposed architecture for ProduceHits

Hierarchy: song → sections → phrases (4/8 bars) → bars → half-bars → beats. Each level constrains the one below it with a prior and passes a confidence up.

Green = sacred input, blue = new CPU code, orange dashed = optional GPU step.

Bar-level chord decoding (sketch)

for each bar b:   x_b      = mean(beat chroma in bar b)          # 12-d, RMS weighted
                  x_b,h    = [mean(first half), mean(second half)]
                  x_b,beat = beat chroma (4)
score(chord c, span s)     = cosine(x_span, template_c)           # triads now, 7ths later
cost(bar b, hypothesis h)  =  - sum_span score  +  lambda_change * (#changes in bar)
                             + lambda_phrase * [change not on phrase bar 1 or bar 3]
Viterbi over bars with transition cost  T(c_prev, c) = 0 if same, mu if diatonic to key, mu + nu otherwise
choose finer split (2 or 4) only if cost drops by more than a margin  =>  default is 1 chord per bar
tie repeated phrases: bars in the same section-cluster share chord posteriors (soft sum)

Parameters (lambda_change, lambda_phrase, mu, nu) are fitted by grid search on Billboard/Isophonics-style annotations, per genre bucket. design proposal

“Don’t over-segment” rules

Minimum section length 4 bars; boundaries only on SANCT bar lines.
Cap sections to roughly one per ~8 bars on average; merge neighbours whose bar-SSM similarity exceeds a threshold.
Keep a boundary only if supported by novelty above a threshold or by two independent voters (SSM, allin1, SongFormer).
Chords: minimum duration of one beat, default one per bar; show chords faded by confidence (already the product guidance from 2026-10-09).
Low-confidence songs (odd meter, no stable bar phase, very short) return whole-song key + beat-level chords only, labelled as such.
Don’t force Verse/Chorus words. If the label model and the SSM disagree, show neutral labels (A, B, C… or Low/Rising/High energy for beatless material).

6a · Recommendation table

Library	Task	Accuracy (reported)	Licence	Speed	Verdict
librosa (ours)	chroma, SSM, clustering	n/a (feature tool)	ISC unv.	CPU, seconds	USE backbone of phases 1–2
mir_eval (pinned)	scoring: segment, chord, key, beat	n/a	MIT unv.	trivial	USE benchmark harness
allin1	sections + labels + downbeats	HR.5F .60–.66, ACC .74 (Harmonix)	MIT code; weights unstated	~27 s of audio/s on 4090; ~9–12 s/song L40 (SongFormer README)	PILOT on Modal T4 after weights-licence check
SongFormer	sections + labels	HR.5F .70, HR3F .78, ACC .81	Code CC-BY-4.0; MuQ weights CC-BY-NC	2–4 s/song on L40 (README)	GATE evaluate offline; no prod until licence cleared
MSAF	classical segmentation	older, unmeasured here	MIT	CPU	REFERENCE ablation baseline only
Beat This!	beats/downbeats	beat F1 89.1, downbeat F1 78.3	MIT code+weights	~20 M params; CPU speed unv.	DIAGNOSTIC cross-check of SANCT bar phase only
madmom	beats, chords, key	various	models CC BY-NC-SA	CPU	AVOID in prod (NC models; install friction)
Chordino / NNLS	chords	unverified	GPL-2.0	CPU	AVOID linking; could be an offline reference only
Essentia	key profiles, chords	unverified	AGPL-3.0	CPU	AVOID as a dependency; borrow the idea of edma/edmm/bgate profiles (re-derive from papers)
BTC	chords (maj/min, large vocab)	baseline in ChordFormer table	MIT code; weights unv.	small model	PILOT as a learned alternative to templates
ChordFormer	chords (large vocab)	MajMin 84.1, 7ths 72.3	release unv.	unv.	WATCH check for released weights
Key CNN (Korzeniowski)	global key	GiantSteps w 74.6; Billboard 85.1	unv.	small CNN	WATCH use paper as the benchmark to beat
Spotify audio analysis	sections	n/a	closed to new apps	n/a	NO

6b · Ablations and benchmark plan

Ablations

Key: full-mix HPSS (today) vs harmonic stem vs bass+other stems vs no HPSS.
Key: KK vs Temperley vs edma/edmm-style profiles vs an ensemble vote.
Key: flat global vs pooled-by-repeat-cluster vs bar-level key HMM.
Key: with and without per-track tuning estimation.
Sections: equal slices (today) vs energy novelty (demo) vs bar-SSM vs allin1 vs SongFormer vs 2-of-3 fusion.
Sections: effect of snapping to SANCT bars on HR.5F.
Chords: beat argmax (today) vs bar Viterbi vs bar+half-bar+beat hypotheses vs + phrase prior vs + key prior.
Bar phase: SANCT downbeats vs Beat This! downbeats vs allin1 downbeats (agreement and chord accuracy impact).

Datasets and metrics

Keys: GiantSteps Key (Beatport previews, electronic) and Billboard-type sets as used in the key-CNN paper; metric mir_eval.key.weighted_score plus the raw correct/fifth/relative/parallel split.
Chords: Isophonics/Beatles, McGill Billboard, RWC-Pop; mir_eval.chord root, majmin, sevenths WCSR; plus our own chords-per-bar and flicker rate.
Sections: Harmonix Set / SongFormBench (200 Harmonix + 100 Chinese songs), SALAMI, RWC-Pop; mir_eval.segment detection P/R/F at 0.5 s and 3 s, pairwise F, V-measure.
Beats: Harmonix beats/downbeats; mir_eval.beat F, CMLt, AMLt, downbeat F.
Internal set (new): 100–150 tracks from the founder’s own catalog and licensed/public-domain audio, hand-labelled for key, modulations and sections, balanced across pop/rock, hip-hop, house/techno, orchestral/other.

Most public sets distribute annotations only; audio must be sourced separately. Keep any benchmark audio private and out of the repo and out of demos.

6c · Compute and licence notes

Where it runs: everything is worker-side. Phases 1–2 are CPU on the auto-scaled Hetzner workers, adding an estimated <1–2 s per song on top of the measured ~13.6 s / ~1 GB baseline (estimate: SSM on ~100–200 bars and a 24-state Viterbi are tiny). The optional deep step is GPU-only on Modal T4. The API origin never touches audio.
Demucs reuse: allin1 runs Demucs internally. We already produce stems for paid users, so a pilot should check whether stems can be passed in to avoid a second separation unverified. Free-tier songs have no stems; the CPU path must work from the mix.
Pinned versions: requirements.txt has torch 2.6.0 while modal_gpu.py pins torch 2.4.1; SongFormer’s requirements pin torch 2.4.0. allin1 needs NATTEN builds matched to torch/CUDA. Budget integration time for a dedicated Modal image.
Licence flags: (1) MuQ weights CC-BY-NC → SongFormer commercial use unresolved. (2) madmom models CC BY-NC-SA. (3) Essentia AGPL-3.0 and NNLS-Chroma GPL-2.0 are copyleft. (4) Weights of allin1/BTC/ChordFormer are not stated in what I fetched. (5) Beat This! is MIT but notes some training data is copyrighted or under limited CC licences. None of this is legal advice; get a one-hour review before any model ships.
Credits / storage laws: new analysis is part of the existing bpmandkey_analyze job; results go into the existing analysis JSONB on processed_songs (additive fields). No new job type is needed (Track 12 plan). Credit pricing for any extra GPU step must go through cost-guard measurement first.

6d · Phased roadmap

Phase	Deliverable	Effort	Expected gain ESTIMATE (not measured)	Gate
0 · Measure	mir_eval harness; baseline for current key, chord flicker, SANCT downbeat vs Harmonix; internal labelled set	3–5 days est.	A real accuracy number for key (replaces “very accurate”); bar-phase reliability	Datasets sourced
1 · Bar-level chords	Viterbi with 1/2/4-resolution, phrase prior, key prior; behind a flag, same JSON shape plus `bar`	1 week	Large drop in chord changes per bar (target ≤1.2 on pop); MajMin gain of a few points likely est.	Phase 0 metrics
2 · CPU sections + pooled key	Bar-SSM sections snapped to bars, repeat clusters, pooled per-section key, key HMM, modulation flag	1–2 weeks	Per-section keys meaningful; whole-song key +1–3 pts on mixed material est., unproven; detect final-chorus modulations	Phase 0 + 1
3 · Deep second opinion	allin1 and SongFormer offline eval on Modal T4; fusion rules; licence decision	1–2 weeks	Boundary F at 0.5 s from ~0.5–0.6 (CPU, est.) toward 0.65–0.70; real labels	Licence cleared
4 · Better chords	Learned model (BTC / ChordFormer if weights are usable) vs templates; 7ths; bass-stem root prior	2–3 weeks	Sevenths and inversions; at most low single-digit WCSR over a tuned template Viterbi est.	Phase 1 baseline
5 · Product surface	Section bands, per-section key chips, modulation badge, confidence fading, CSV with bars	1 week	Trust & usability	ux-polish gate

07Honest risks

Label ambiguity

Functional labels (verse/pre-chorus/chorus/bridge) are subjective and schema-dependent. SongFormer’s whole premise is training with “schema-mismatched labels”. Expect disagreement even among experts; ship neutral labels when models disagree.

Electronic and loop-based music

Sections are defined by layers and energy, not by harmony. Chroma-SSM finds a loop repeated for 64 bars and calls it one section. Key profiles tuned on pop fit electronic worse (key-CNN papers). A single-chord or drone track makes chord Viterbi and key pooling nearly meaningless.

Tempo changes and time signatures

SANCT assumes stable bar groups of 4 beats. 3/4, 6/8, 5/4, meter changes and rubato break the bar tensor. Beat This! authors themselves note failures on non-periodic pieces. Detect and fall back.

Lyrics-driven structure

Verses often share a melody and chords but differ in lyrics; choruses can be defined by hook text. Audio-only models partially capture this. Gemini-style lyric-aware models score higher at ±3 s but worse strict boundaries (SongFormer table).

Over-segmentation and false modulations

Every extra boundary or key flip that a user sees and finds wrong costs more trust than a missed one. Hence the 4-bar floor, the 8-bar evidence rule and the whole-song fallback.

Benchmark and licence risk

No existing key ground truth; models trained on Harmonix may overfit pop/EDM; some weights are non-commercial. Estimated gains in this report are not measurements until phase 0 runs.

—Claim ledger (decision-driving claims, re-checked 2026-10-18)

Each source was re-opened (paper PDF/abstract, repo README or LICENSE, product page) and the number or licence compared with the claim. “Fixed” = the report text was corrected or softened in this validation pass.

Claim in report	Label	Source check	Status
SongFormer HR.5F .703, HR3F .784, ACC .807 on SongFormBench-Harmonix	MEASURED	SongFormer README table (also arXiv 2510.02797): HX variant .703/.784; ACC .807 belongs to the HX+E+H+G variant	Confirmed; variant caveat added
allin1 Harmonix 8-fold: beat F1 .958, downbeat .915, HR.5F .660, label PWF .738	MEASURED	arXiv 2307.16425 Table 1 (All-In-One 300K: .958 / .915 / .660 / .738)	Confirmed
allin1 on SongFormBench-Harmonix: HR.5F .596, HR3F .730, ACC .740	MEASURED (by SongFormer authors)	SongFormer README	Confirmed
allin1 speed 33 min audio in 73 s on RTX 4090	MEASURED (README)	allin1 README	Confirmed
SongFormer 2–4 s/song on L40; allin1 9–12 s	MEASURED (SongFormer authors, README)	README speed table (excl. model load)	Confirmed; GPU is L40, not our Modal T4 unverified on T4
ChordFormer MajMin 84.09, Sevenths 72.28, Root 84.69, MIREX 83.62; baseline CNN+BLSTM 82.62/69.78	MEASURED	arXiv 2502.11840 Table II; dataset 1,217 songs (Isophonics, Billboard, MARL), 5-fold CV, WCSR	Confirmed; dataset/CV wording added. Earlier note that the baseline row was “83.4/82.6/69.8/81.5” matches
Beat This! beat F1 89.1, downbeat F1 78.3 (77.4 with DBN)	MEASURED	arXiv 2407.21658 Table 2 (GTZAN test set)	Fixed: dataset was mislabelled as “broad 8-fold set”; removed unverified “78 MB”
Key-CNN GiantSteps w 74.6 / Billboard 85.1 / Classical 96.6	MEASURED	arXiv 1808.05340 results table	Confirmed
Local key needs whole-piece harmonic coherence	verified	1808.05340 abstract	Confirmed; now cited in 4b
Pop-tuned key systems underperform on electronic; errors are relative/parallel	verified	arXiv 1706.02921 abstract	Confirmed
MuQ weights CC-BY-NC 4.0, code MIT	verified	MuQ README “License”	Confirmed. Gap: SongFormer’s exact checkpoint not traced, so “commercial blocker” softened to “likely”
madmom models CC BY-NC-SA 4.0; code BSD	verified	madmom LICENSE file	Confirmed
NNLS-Chroma GPL-2.0; Essentia AGPL-3.0	verified	COPYING files in each repo	Confirmed. The “AVOID” verdicts are our legal-risk judgement, not legal advice
allin1 / BTC / Beat This! / MSAF code MIT	verified	GitHub licence API / LICENSE / Beat This! README (MIT code and weights; notes some training data copyrighted)	Confirmed
Spotify Audio Analysis closed to new apps	verified	Spotify 2024-11-27 blog	Reworded to match the post exactly
rekordbox auto cues use Phrase analysis; skipped if phrases cannot be analysed / track too short	verified	rekordbox 7 FAQ	Confirmed
Moises Chord Finder, no published accuracy number	verified	moises.ai page: only “most users find… quite accurate”	Confirmed
Our pipeline: ~13.6 s and ~0.9–1.1 GB per song	MEASURED (internal)	docs/reports/2026-10-09-song-intelligence-strategy.md table (13.6 s; 888 MB / 1,065 MB peak)	Confirmed
Phase 1–4 expected gains; +<1–2 s CPU; CPU boundary F 0.5–0.6; whole-song key +1–3 pts; chords/bar ≤1.2	ESTIMATE	No source. Projections only; the phase-0 harness exists to test them	Labelled
Bar snapping raises strict boundary F	ESTIMATE	No source	Labelled
Hierarchical decoding beats shorter windows (4b)	ESTIMATE with partial support	Mechanism stated by ChordFormer and Key-CNN authors; no head-to-head	Stated as hypothesis + falsifying test

Not checked: MIREX results, weights licences for allin1/SongFormer/BTC/ChordFormer, any competitor beyond the three listed (see Unverified list).

—Sources

All fetched on 2026-10-18 unless noted.

Kim & Nam, All-In-One Metrical and Functional Structure Analysis (WASPAA 2023): arxiv.org/abs/2307.16425; code github.com/mir-aidj/all-in-one (MIT).
SongFormer: arxiv.org/abs/2510.02797; code github.com/ASLP-lab/SongFormer (CC-BY-4.0 code).
MuQ: github.com/tencent-ailab/MuQ (code MIT, weights CC-BY-NC 4.0). MusicFM: github.com/minzwon/musicfm.
MSAF: github.com/urinieto/msaf (MIT).
Beat This!: arxiv.org/abs/2407.21658; github.com/CPJKU/beat_this (MIT).
Beat Transformer 2209.07140; SpecTNT 2205.14701; BEAST 2312.17156 (abstract-level only).
madmom: github.com/CPJKU/madmom, LICENSE (code BSD-style; models CC BY-NC-SA 4.0).
ChordFormer: arxiv.org/abs/2502.11840. Chord Recognition with Deep Learning: arxiv.org/abs/2512.22621. BTC: arxiv.org/abs/1907.02698, code (MIT).
NNLS-Chroma/Chordino: github.com/c4dm/nnls-chroma (GPL-2.0).
Essentia: github.com/MTG/essentia (AGPL-3.0); KeyExtractor.
Korzeniowski & Widmer, Genre-Agnostic Key Classification with CNNs: arxiv.org/abs/1808.05340; End-to-End Musical Key Estimation: arxiv.org/abs/1706.02921.
Spotify Web API changes, 2024-11-27: developer.spotify.com.
rekordbox 7 FAQ (phrase analysis): rekordbox.com. Moises Chord Finder: moises.ai.
Repo (internal): afcut_engine/harmonic.py, webapp/app.py (_detect_key, _harmonic_layer), webapp/beat_engine.py (read-only), tools/song_intel_demo.py, docs/plans/2026-09-14-track12-bpmandkey-harmonic-analysis.md, docs/reports/2026-10-09-song-intelligence-strategy.md, requirements.txt, modal_gpu.py.

—Unverified list

MIREX structure, chord and key results (music-ir.org not reachable from the research environment).
Weights licences for allin1, SongFormer, BTC, ChordFormer; whether allin1 loads madmom model files at runtime.
Accuracy of Chordino, madmom chords, Essentia, autochord, CREMA; any 2024–26 chord or key model other than ChordFormer and the 2025 thesis.
Commercial tools beyond rekordbox, Moises and Spotify: Mixed In Key, Serato, Ableton, Logic, Cubase, Chordify, Songsterr, Cyanite, Musiio, AudioShake, LANDR, Beatport.
Inter-annotator agreement for sections and chords; harmonic-rhythm statistics by genre; frequency of pop modulations; any dedicated modulation-detection literature.
Harmonix Set genre composition; librosa, mir_eval, Demucs licences (from memory); Beat This! CPU speed; allin1 stems pass-through; T4 vs L40 speed for SongFormer.
Whether our CQT path estimates tuning; accuracy of SANCT downbeats; the two calibration tracks’ details.
All “expected gain” figures in the roadmap are estimates, not measurements.

Repo	Licence (GitHub)	Last push	Stars	Open issues*	Notes
mir-aidj/all-in-one	MIT	2024-05-09	862	20	Last commit 2024-05-09; open issues/PRs run into 2026, so the community is live (not abandoned). Open: NATTEN incompatibility (#30, PR #36 Aug 2025 / PR #39 Sep 2026), “uses too much CPU” (#38), commercial use question (#40 Oct 2026), Harmonix label inaccuracies (#34), RTX 5090 incompatibility (#35). Weights licence unverified.
ASLP-lab/SongFormer	NOASSERTION	2026-05-14	181	6	Active-ish. #9 checkpoint licensing unanswered; #5 duplicate section labels; PR #8 VRAM work in draft.
urinieto/msaf	MIT	2026-05-13	560	22	Classical algorithms (SF, Foote, C-NMF). No chords or key.
CPJKU/beat_this	MIT	2026-05-28	428	4	Beats/downbeats only. Possible cross-check for the SANCT grid, not a replacement.
jayg996/BTC-ISMIR19	MIT	2020-05-23	215	10	Chord model, 6 years untouched; training data licence unverified.
CPJKU/madmom	NOASSERTION	2026-03-20	1727	81	Models CC BY-NC-SA (Research tab).
tencent-ailab/MuQ	MIT (code)	2025-08-04	390	3	Weights CC-BY-NC per Research tab.
MTG/essentia	AGPL-3.0	2026-10-14	3775	443	AGPL: do not link into the product.

Item	What it is	What it is NOT
#34 Label inaccuracies in the Harmonix Dataset	One unresolved reporter questions Harmonix section annotations ANECDOTAL	Not proof of dataset-wide label inaccuracy; no measurement, no sample size
#35 use of the NVIDIA GeForce RTX 5090	torch 2.5 + cu121 / RTX 5090 unsupported compatibility report	Not an accuracy result; says nothing about model quality
PR #36 (Aug 2025) NATTEN compatibility fix and working Dockerfiles	An open, unmerged proposed fix	Not merged, not released, not verified by us
PR #39 (Sep 2026) pure-PyTorch CPU/MPS neighborhood attention, NATTEN optional	Unmerged proposal for a CPU/MPS fallback	Two proposed fixes (#36/#39) plus 2026 activity show a live community, not proof the project is maintained or that a release ships
#40 (Oct 2026) commercial use in an iOS app	An unanswered licensing question	Not proof permission was denied or granted; the weights licence stays unverified

#	File (existing)	Change	Needed?	Risk
T0	none	Poller inside the new job: a periodic `song_structure_scan` picks `processed_songs` rows with no `song_structure` row.	Alternative to T1	Low. Zero edits to the SANCT path. Costs up to a poll interval of latency.
T1	`shared/worker_daemon.py`	Register the new job type’s handler in the dispatcher (an import and a mapping entry). Required in any option: the daemon is where jobs are claimed.	Yes	Medium. Shared by every job; a bad import would break all workers. Mitigate: lazy import inside the handler, a unit test that the daemon imports, and the existing kill switch pattern (env flag).
T2	`shared/scaler.py`	Add `song_structure` to `LIGHT_JOB_TYPES` (so it never rents a worker-hour alone, same reasoning as `track_preview`). Without this the job either never wakes a cold fleet or, if put in HEAVY, spends worker-hours.	Yes	Medium. Wrong class = stuck queue or a billing leak. Choose LIGHT and run the cost-guard check.
T3	`shared/models.py` + new alembic revision	New `song_structure` model/table only (db-migration skill). No change to existing models.	Yes	Low if additive. Not created now.
T4	`webapp/app.py`	One read-only GET route and, for any paid tier, a `CREDIT_COSTS` entry. Router could live in a new file mounted by one line.	Yes	Low. Reads a new table. Must not compute (origin never processes).
T5	`webapp/static/js/bpmandkey.js`, SongChop view	Render the lanes; fetch is additive.	Yes (UI)	Low-medium. Must obey the frontend law and not alter loop cut logic.
T6 (optional)	end of the `full_pack` handler	One enqueue line instead of the poller.	No: choose T0	Medium. It sits next to SANCT code; avoid.

Law	How the design complies
API origin enqueue-only	Compute only inside the queued job on an auto-scaled worker. The GET route reads a table. No BackgroundTasks, no subprocess on the origin.
Credits law	Cache by `audio_hash` (unique key on the new table); idempotent `idem:` ref_id if ever charged. Recommended: free by design like `track_analyze`, cost shown as “Already analysed”. Result persisted in Postgres.
Storage routing	Reads the stored FLAC from StorageBox (never served to browser). Result is JSON in Postgres; no audio written, no new WAV/FLAC.
SANCT law	No change to cut points or slicing. SongChop uses sections only as labels and suggestions over existing bar indices.
DB law	New table via alembic (db-migration skill), additive, reversible.

Option	Extra CPU-s	Peak RAM	Wall	€/song	Basis
Current BPM&Key full analysis	~13.5	0.9–1.1 GB	~14 s	~€0.0001	MEASURED (strategy report)
Own analyzer, CPU (fresh chroma pass, shares decode)	+4 to +8	+0.3 to +0.8 GB (child process, same cap)	+4–8 s	+€0.00003–0.00006	ESTIMATE: existing chord step is 2.6 s, section-key 2.6 s, our synthetic maths 1.8 s (I6); real CQT costs more than the prototype’s STFT
Own analyzer, reusing saved harmonic features	+1 to +3	~+0.3 GB	+1–3 s	<€0.00003	ESTIMATE. Needs the chroma to be saved by the existing path, which would add a touch point; not assumed.
allin1 on CPU	60–200	2–4 GB	1–3 min	€0.0004–0.0014	ESTIMATE. Not benchmarked. Issue #38 reports heavy CPU use (title read only). Whether it re-runs stem separation by default: unverified.
SongFormer on Modal GPU (T4)	GPU 5–20 s + cold start	GPU VRAM several GB (PR #8 is about VRAM)	10–60 s with cold start	~€0.002–0.01	ESTIMATE. T4 hourly rate and model speed not checked. Needs a gateway job and licence clearance.

Stage	Seconds
STFT	0.99
Beat-sync chroma	0.09
SSM + novelty	0.002
Chord Viterbi (120 bars × 24 states)	0.002
Per-section key	0.01
Process CPU total (incl. signal synthesis)	1.78
Peak RSS	877 MB

Product	Consumes	Shows
BPMandKey	sections[], key_segments[], chords per bar	Structure timeline, per-section key chips (with “modulates to …” marker), chord-per-bar strip, confidence shading. Whole-song key stays as today, and is shown next to the sectioned one so users can compare.
SongChop	sections[] edges (bar indices), existing loops	Section bands over the waveform; “suggest loops from the chorus” picks existing SANCT bar ranges inside a section. Never writes cut points, never moves a loop boundary; a suggestion is a pair of bar indices handed to the existing loop code.
SongTranscribe	chords per bar	Cleaner chord chart (Research tab’s main win).
Remix / PodDJ / others	sections	Arrangement blueprints: the strategy report names remix blueprint prep as a consumer ESTIMATE (not scoped here).

Failure mode	Design mitigation
Key wrong on electronic / modal / ambiguous music	Return top-3 with margin; allow “indeterminate” (the existing engine already has an indeterminate path); show confidence, never a bare label.
Relative major/minor confusion	Always output the relative key and its margin; disambiguate with chord-root prevalence and bass/tonic-chord evidence per section; mark low margin visibly.
Over-segmentation	Minimum section length of 4 bars, edges only at downbeats, novelty threshold adaptive to the song, merge sections with the same repetition cluster; hierarchy (phrase → section).
Chord flicker	Bar-level states, high self-transition prior, beat-level change only when evidence is strong, majority pooling across repeats of a section.
Wrong downbeat (bar shifted by 1–3 beats)	Sidecar cannot fix SANCT. Correct SANCT beats do not automatically establish downbeats or bar phase, so the phase-0 audit must verify downbeats independently — without altering the grid. Detect inconsistency (chord changes landing mid-bar systematically) and flag `grid_confidence` low; offer a UI “bar offset” that only shifts analysis labels, never audio cut points.
Tempo changes / free tempo	Work in bars from the grid; if beat intervals drift beyond a threshold, split into segments and analyse each, or mark “unsupported”.
Modulation false positives	Require a sustained change over ≥ 8 bars, win by margin, and relative-key/chord support before showing a modulation.
Stale or corrupt result	Version field; recompute on mismatch; `status: failed` shown as “analysis unavailable”, never blocking SongChop.

Phase	Scope	Effort ESTIMATE	Gate to proceed
0 Benchmark & ablation	Private labelled set (Harmonix annotations plus ~30 own tracks, audio kept out of repo). Score: boundary F, chord WCSR, key accuracy. Ablate hierarchy vs shorter windows (open question from the Research tab), own segmenter vs allin1 vs SongFormer (offline only), audit SANCT downbeat accuracy.	3–5 days	Own segmenter within ~10 pts of the best on HR3F, or a clear plan; licence decision for SongFormer.
1 Sections	Sidecar job, table, migration, touch points T1–T4, GET route, BPMandKey timeline. Bar-snapped sections, cluster labels, unknown allowed.	5–8 days	Beta songs pass visual review; queue and memory unchanged (measured p50, p95).
2 Chords	Bar-level Viterbi, simplified/extended toggle, chord strip UI, SongTranscribe chart.	4–6 days	Flicker rate and WCSR at or above current chord timeline on the benchmark.
3 Per-section key / modulation	Pooled section key, relative-key margin, modulation marker.	3–5 days	Beats whole-song key on a modulating subset; no regression on the rest.
4 Per-stem (later)	Bass-stem root evidence for chords and key, drums for downbeat cross-check. Needs stems (paid, GPU). Not started until 1–3 prove value.	1–2 weeks	Measured gain vs the extra stem cost.
Backfill	See I10; after phase 1 gate.	1–2 days	Cost-guard sign-off.

Layer	Contains	Depends on	Cache key	Cost for a 4-min song
A · grid-independent	frame CQT / chroma, tuning estimate, frame novelty / self-similarity, onset envelope	audio frames only	(audio_hash, analyzer_version)	ESTIMATE 3–8 CPU-s, 0.3–0.8 GB. Anchor: the existing chord step is 2.6 s and per-section key 2.6 s MEASURED (strategy report)
B · grid-dependent	beat-sync chroma aggregation, bar/phrase grouping, meter segments, bar-level chord Viterbi, section segmentation, per-section key	Layer A + the grid revision	(audio_hash, grid_revision_hash, analyzer_version)	ESTIMATE ~0.1–2 CPU-s. Anchor: prototype decision stages are 0.002–0.09 s ESTIMATE (timing only; the prototype decoder collapsed to one chord, I6)

Edit	Layer A	Layer B scope
Move the 1 (phase)	reuse	bar grouping + bar-level chord Viterbi + section edges. Range-scoped if the edit is range-scoped, whole-song if applied whole-song.
Move one beat (`apply_single_beat_override`)	reuse	beat-sync aggregation at that beat; the function moves one beat and leaves the rest, so there is zero downstream drift.
Inpaint from anchor (`inpaint_from_anchor`)	reuse (audio is read, never written)	downstream of the anchor: bar grouping, bar-level chords and section edges from that point forward; upstream beats stay frozen verbatim.
Insert / delete beats	reuse	beat-sync aggregation from the edit forward (bar indices shift downstream).
Set meter for a range (3/4, 2-beat interlude)	reuse	meter segments + phrase/bar grouping for the range, plus the boundary sections.
Mark gap / skip zone (e.g. a 2-beat turnaround)	reuse	exclude the region; section segmentation may need one global pass because the repetition structure changed.
Tempo change (beats unchanged)	reuse	bar lengths and tempo-dependent features only.
Tempo edit that moves beat times	reuse (frames unchanged)	beat-sync aggregation and everything downstream, for the range.
Analyzer version bump	recompute	recompute.
New or changed audio	recompute	recompute.

Symbol (verified, read-only)	What it does	Audio or grid
`webapp/beat_inpaint_engine.py:283` `inpaint_from_anchor(...)`	Freezes upstream beats verbatim, re-detects downstream from an anchor, returns `beat_times`/`downbeat_times`/`tempo`.	Grid only. Audio is read and sliced in memory for analysis; nothing is written back.
`webapp/beat_inpaint_engine.py:434` `apply_single_beat_override(...)`	Moves one beat index.	Grid only (array edit).
`webapp/beat_inpaint_engine.py:452` `inpaint_with_skip_zones(...)`	Indexes beats around skip zones (2-beat fills).	Grid only.
`webapp/app.py:25152` `POST /api/inpaint-beats`	Returns the welded arrays; does not persist.	Grid only.
`webapp/app.py:25014` `POST /api/packs/{pack_id}/save-cuts` (`save_pack_cuts`)	Writes manifest keys and patches `processed_songs.analysis`; docstring: “without re-extracting samples or demos”.	Grid / metadata only.
`webapp/app.py:38110` `PUT /api/user/song-edit`	Persists `offset, beatRate, one, downbeat_anchor, micro_nudge_sec, inpaint_anchors, skip_zones`.	Grid / metadata only.
`shared/beat_review.py`	Versioned human attestation. Docstring: “Personal listening attestations. Never changes the detector or audio.”	Attestation only.
`shared/models.py:229` / `:551` `beat_verified`	Boolean flag on `ProcessedSong` / `UserTrack`.	Metadata.
`webapp/app.py:25276` `_do_recut_sync` → worker `_handle_recut`	Re-derives cut points and re-slices audio at `int(round(t*sr))`.	Audio. The only audio-mutating path; it runs on workers, and a metadata edit must never trigger it.

Scenario	Layer A	Layer B	Decision	Cost ESTIMATE
First analysis of a new upload	compute	compute all	Full	~13.5 CPU-s MEASURED (existing BPM&Key)
Same audio + same grid revision, cached	reuse	reuse	Cache hit (no job)	0
Human confirms the grid, no change (attestation)	reuse	reuse	Cache hit (no job)	0
Move the 1 over a range	reuse	rerun affected bars + downstream section edges	Layer B (local; whole-song if applied whole-song)	~0.1–2 CPU-s
Insert / delete beats	reuse	rerun from the edited bar forward	Layer B (local)	~0.1–2 CPU-s
Set meter for a range	reuse	rerun the range + boundary sections	Layer B (local)	~0.1–2 CPU-s
Mark a gap / skip zone	reuse	rerun the range; one global section pass	Layer B (local + one global)	~0.2–3 CPU-s
Tempo map edit (beats unchanged)	reuse	bar lengths / tempo-dependent features	Layer B (local)	<1 CPU-s
Tempo edit that moves beat times	reuse	beat-sync aggregation + downstream, for the range	Layer B (range)	~0.1–2 CPU-s
Audio re-decoded for a range only (partial re-upload / repaired region)	recompute region	recompute region	Region	~1–3 CPU-s per region ESTIMATE
Analyzer version bump	recompute	recompute	Full	~13.5+ CPU-s
Audio changed (new decode / different upload)	recompute	recompute	Full	~13.5+ CPU-s

Representation	4-min song @ 44.1 kHz, hop 512 (~20.7k frames)	8.1k-song library
12-bin chroma, float32	~1.0 MB	~8 GB
12-bin chroma, float16	~0.5 MB	~4 GB
84-bin CQT, float32	~7.0 MB	~57 GB
84-bin CQT, float16	~3.5 MB	~28 GB

Evidence in report	Label	Our plan	Status
rekordbox: “Phrase analysis” marks track phrases; auto hot-cue/memory-cue placement uses it, and is skipped when phrases cannot be analysed or the track is too short.	verified rekordbox 7 FAQ	Self-similarity sections snapped to SANCT bar lines (phases 0–2).	planned; unproven until Phase 0
Moises “Chord Finder”: chords synced to playback, Easy/Medium/Advanced voicings, capo mode, editable, exportable chart. No accuracy figure is published on the page.	verified moises.ai	Bar-level chord decoding with a chord strip.	planned; unproven until Phase 0
Spotify audio analysis: removed for new apps and for development-mode apps without a pending extension request, per the 2024-11-27 notice. Apps that already had access are unaffected. Not an option for us.	verified Spotify Web API changes, 2024-11-27 (the report's ledger lists this as the Spotify blog)	None. Not used.	not planned: closed to new apps
Mixed In Key, Serato, Chordify and others. Not fetched. Our own earlier note (2026-10-09) describes MIK/rekordbox/Splice features “based on established product knowledge”. Chordify’s site returned 403 to automated fetch.	unverified see unverified list	No comparison made; the report has no evidence for these tools.	planned; unproven until Phase 0

Stage to roadmap mapping. Gate text in column 4 is quoted from the repo. Column 5 is this memo’s own gate, a design proposal.
Stage	6d phase	6d gate text	Memo gate (design proposal)
S0	None (pre-roadmap)	None	None. No user-visible change.
S1	0 Measure	“Datasets sourced” scout-read	Calibration data from phase 0 sets the thresholds.
S2	1 Bar-level chords	“Phase 0 metrics” scout-read	Bar-level metrics in hand.
S3	None direct	None	Surface confirmed, fetch path chosen, S2 stable for one engine change.
S4	2 CPU sections + pooled key; 3 Deep second opinion	“Phase 0 + 1”; “Licence cleared” scout-read	I12 criterion (quoted at index.html:926) and held-out boundary metric within the agreed gap.
S5	5 Product surface	“ux-polish gate” scout-read	Validated labeller and ux-polish pass.

Control	Values	Where	Tag
Beat rate (ratio p:q)	8 presets: none (1:1); 3:2 "Dotted→Straight"; 6:2 "Dotted→Straight 2×"; 2:1 "Half→Quarter"; 4:1 "Whole→Quarter"; 3:1 "Dotted Half→Quarter"; 1:2 "Eighth→Quarter"; 2:3 "Triplet→Straight"	`webapp/static/js/segments.js:4-14` (`BEAT_RATE_PRESETS`). Dropdowns at `webapp/static/index.html:1017-1026` (Chop) and `:1468-1477` (Pack).	VERIFIED IN REPO (presets). SCOUT-READ (dropdown lines).
Ratio use note	6:2 is meant for when the detector locked onto off-beats. Its clicks run twice as fast, so the offset can land them on the real pulse.	`segments.js` code comment next to the 6:2 preset.	VERIFIED IN REPO
Derivation `correctBeats(beatTimes, ratio)`	Groups every q detected intervals and splits each group into p equal parts. Returns a new array. The input is not changed. Example: 3:2 turns every 2 detected beats into 3 evenly spaced beats.	`segments.js:18` (`correctBeats`, lines 16-34 per brief).	VERIFIED IN REPO
Beat shift (offset)	Whole beats. Clamped to −16…+16 on the server.	Client: `segmentGridStartIndex(...)` returns `phase + offset` (`segments.js:48-72`). Server clamp: `webapp/app.py:31308`, `:33084`.	VERIFIED IN REPO (client). SCOUT-READ (server clamp).
Downbeat / bar phase	Voted by snapping SANCT `downbeat_times` to the nearest corrected beat, then counting `index % measureBeats`. `measureBeats = 4` is hard-coded.	`segments.js:48-72` (vote), `segments.js:80` (`measureBeats`).	VERIFIED IN REPO
Downbeat offset (founder's manual control)	This is the beat shift applied on top of the voted phase. It is not a separate control.	`segments.js:48-72`.	VERIFIED IN REPO
Beats per segment `bps`	Default 8. Sets the loop length in beats.	`calcSegments(rawBeats, offset, bps, beatRate, …)` at `segments.js:74`. Meaning is from code reading.	VERIFIED IN REPO (signature). SCOUT-READ (meaning).
Skip zones (turnaround fills)	Supported. Fills such as 2-beat fills can be skipped.	`segments.js:~90-120` (`skip_zones`).	VERIFIED IN REPO
Initial/default editor values	ratio none, shift 0, phase from SANCT downbeat vote	SCOUT-READ/HYPOTHESIS: confirm where defaults are set	UNVERIFIED (where defaults are initialised)
Meter-change or interlude detection	None exists.	No code found.	HYPOTHESIS (later work)

ILLUSTRATIVE — invented 0–1 scores. Not measured.
Case	Default reading (SANCT 1/1)	Candidate A	Candidate B (best)	Evidence that moves the score	Margin → what the editor opens with
1. Polyrhythm	1/1 shift 0: 0.31. Section lengths come out at about 5.3 bars. HYPOTHESIS	2:1 shift 0: 0.44	3:2 shift 0: 0.88	Whole 8-bar sections. Boundaries on bar lines.	Margin 0.44, clear. Editor opens at 3:2. ESTIMATE confidence high
2. Off-beat lock	1/1 shift 0: 0.35. Chord changes land on beat 2 or 4. HYPOTHESIS	2:1 shift 0: 0.36	2:1 shift 1: 0.83	Chord changes on beat 1. Section edges on 8-bar multiples.	Margin 0.47, clear. Editor opens at 2:1 + shift 1. ESTIMATE confidence high
3. Combo	1/1 shift 0: 0.22	3:2 shift 0: 0.66 (runner-up)	6:2 shift 1: 0.71	Both readings keep 8-bar sections. Chord changes split between them.	Margin 0.05, small. Below threshold: editor opens at today's default (1/1). Case logged for review. ESTIMATE
4. Downbeat far off	Phase 0 (default): 0.29	Shift 2: 0.41	Phase 3: 0.86	Downbeat vote over offsets 0 to 3. Onset and low-band energy peak on phase 3.	Margin 0.45, clear. Editor opens with downbeat +3. ESTIMATE confidence high

Question	Answer	Tag	Where
Which beat path runs?	`detect_beats_unified` runs v11 by default or lite. It falls back to `_detect_beats_sanct`.	SCOUT-READ	`webapp/beat_engine.py` (~:729, fallback ~:566). Line numbers drift.
How is tempo found?	Three parallel probes. Each is folded into 70–140 BPM by ×2 or ÷2. The median is taken.	SCOUT-READ	`webapp/beat_engine.py` (~:482–502, scout-read; lines drift)
How are beats placed?	`librosa.beat.beat_track` with start_bpm = tempo, tightness 200 or 400, trim=False.	SCOUT-READ	`webapp/beat_engine.py` (~:535–543)
How are downbeats picked?	Each beat scores 0.6 × onset + 0.4 × low-band RMS. Four phase sums are compared. The argmax picks every 4th beat.	SCOUT-READ	`_detect_true_downbeats` in `webapp/beat_engine.py`:135; phase vote :187–192 (VERIFIED IN REPO)
What is the bar length?	`_detect_true_downbeats(audio, sr, beat_times, measure_beats=4)` defaults to 4 beats per bar (webapp/beat_engine.py:135); its only call site (:553) passes no value; the lite path uses `beat_times[::4]` (:717).	VERIFIED IN REPO	`webapp/beat_engine.py`:135, :553, :717
Is phase checked again?	Yes. `segment_grid_start_index` re-votes phase against the beat grid.	SCOUT-READ	`webapp/beat_engine.py`:196 (VERIFIED IN REPO)
Does it detect meter or swing?	No 3/4 or 6/8 detection. No swing handling. No half-time or double-time choice beyond the octave fold.	SCOUT-READ	`webapp/beat_engine.py`
Segment editor reading (ratio, shift, offset)	Chosen by hand; initial defaults are fixed; no automatic choice of ratio/shift; only downbeat phase is automatic	VERIFIED IN REPO SCOUT-READ	`segments.js:4-87`, `user_grid.py:36-47`
Is there a benchmark or golden test?	No beat or downbeat benchmark. No golden regression test. A glob of `tests/*/beat*.py` found nothing.	SCOUT-READ	`tests/`
Is mir_eval used?	It is pinned but not used.	SCOUT-READ	`requirements.txt` (:58)
What does the chord timeline return?	Entries with start, end, chord, root, quality and confidence. Identical consecutive chords merge. Capped at 200 entries.	VERIFIED IN REPO	`afcut_engine/harmonic.py` `chord_timeline` (:1099)
How is chord confidence set?	1 − exp(−margin / 0.08), clamped to 0–1.	VERIFIED IN REPO	`afcut_engine/harmonic.py`
Is the chord timeline beat-synced?	Yes when beat_times is supplied. Otherwise it uses a free-time fallback.	SCOUT-READ	`afcut_engine/harmonic.py`
What do per-section keys return?	Index, start, end, key, scale, key_label, camelot, confidence and key_ambiguous.	VERIFIED IN REPO	`afcut_engine/harmonic.py` `detect_per_section_keys` (:734)
How are per-section keys cut?	Audio is split into equal time spans. The harmonic layer asks for 4. It never uses beats.	SCOUT-READ	`webapp/app.py` `_harmonic_layer` (~:32343)
Is there a real structure segmenter?	No. The repo has no structural segmenter. This was not searched exhaustively.	SCOUT-READ UNVERIFIED	No snapping search done.
Does the harmonic layer write to beat_times?	It copies beat_times to a new float list before use. No write was found there.	SCOUT-READ	`webapp/app.py` harmonic layer
Does `apply_grid` change beats?	Not checked.	UNVERIFIED	`webapp/app.py` (~:15272, ~:15780)
Which analysis version is written?	analysis_version "3", built in `_build_bpm_key_analysis`. This is additive JSON.	SCOUT-READ	`webapp/app.py`

Result	Value	Label	Source/variant
SongFormer HR.5F	.703	MEASURED	Literature; HX variant
SongFormer HR3F	.784	MEASURED	Literature; HX variant (ACC .807 belongs to the HX+E+H+G variant)
allin1 Harmonix 8-fold HR.5F	.660	MEASURED	arXiv 2307.16425, Table 1
ChordFormer MajMin	84.09	MEASURED	Literature
Beat This! GTZAN downbeat F1	78.3	MEASURED	arXiv 2407.21658 Table 2, GTZAN test set
Key-CNN GiantSteps w	74.6	MEASURED	arXiv 1808.05340

Component	Licence	Label
MuQ	Code MIT; weights CC-BY-NC 4.0	verified
madmom models	CC BY-NC-SA 4.0	verified
Essentia	AGPL-3.0	verified
NNLS-Chroma	GPL-2.0	verified
allin1	Code MIT; weights unstated	unverified
SongFormer	Unresolved; treat as not cleared	unverified

Fact	Where (verified in repo)	Why it matters
analyze_harmonics returns `{key, per_section_keys, character, chord_timeline}`, with no version field inside.	`afcut_engine/harmonic.py:1214` verified in repo	The version has to be stamped by the payload, not read from the engine. The contract needs its own version field.
`chord_timeline` entries carry start, end, chord, root, quality and confidence (0 to 1). Identical consecutive chords are merged. Capped at 200 entries.	`harmonic.py:1099` verified in repo	Confidence already exists per chord, so the display can fade by it. The 200-entry cap can cut off long songs; the display has to show that, not hide it.
`detect_per_section_keys(audio, sr=44100, sections=4)` entries carry index, start, end, key, scale, key_label, camelot, confidence and key_ambiguous. Degenerate sections carry `indeterminate` or `no_usable_signal` with confidence 0.0. Returns `[]` under 2 usable sections.	`harmonic.py:734` (fields at `:776-777`, `:791`) verified in repo (harmonic.py:776-777 and :791 are scout-read)	The sections are fixed windows, not a segmentation. Empty and degenerate states each need their own display state.
Payload is stamped `analysis_version "3"`. v2 rows stay valid, v3 keys are optional, and the data is stored in the `processed_songs.analysis` JSONB.	`webapp/app.py:32427`; `_ANALYSIS_VERSION_CURRENT` at `app.py:458`; `shared/models.py:216-218` verified in repo (shared/models.py:216-218 is scout-read)	Old rows render without a migration. The contract can be versioned additively.
`bpmandkey.js` already renders this data: `_renderResult`, `_keyCard` (label, camelot, confidence meter, three states), `_sectionsPanel` (`.bk-secs` cells for key, time and confidence; text-only “Modulates: A → B”), `_chordPanel` (`.bk-chords`, first 48 then “+N more”, reads chord, start and end but NOT confidence).	`bpmandkey.js:388, :795, :664, :674, :774` scout-read	Most of the display already exists in one file. The chord panel ignores confidence today, so the honest-confidence claim is not visible anywhere yet.
The SongChop player files (`stem-lanes.js`, `library-console.js`, `catalog.js`, `player-tight.js`) read neither `chord_timeline` nor `per_section_keys`. Only `bpmandkey.js` consumes them.	Repo grep verified in repo	The player has to be wired to the shared renderer from scratch.
The library list query selects `ps.beat_times` but not `ps.analysis`. The status path returns `ps.analysis`.	List query is the function at `webapp/app.py:15110`, which selects `ps.beat_times` at `:15153` and does not select `ps.analysis`. `ps.analysis` is selected in `_product_cache_status` (`webapp/app.py:30811`, SELECT `:30822-30823`; returns `row["analysis"]` near `:30852`) verified in repo.	The player needs one extra fetch to get the analysis JSONB.
Beat-grid safety: `chord_timeline` only reads `beat_times`; it builds a new filtered list (`afcut_engine/harmonic.py:1149-1150`, docstring `:1102`). `_harmonic_layer` documents `beat_times` as read-only (`webapp/app.py:32338` docstring, a documentation claim, not a proof). Harmonic output (`per_section_keys`, `chord_timeline`) is stored inside the analysis JSONB. The `beat_times` and `downbeat_times` columns are written by the existing COALESCE upsert (`webapp/app.py:32627-32628`); a reader found no assignment to them from harmonic output in `app.py` 32338-32700 (verified by reading, not by test).	`harmonic.py:1149-1150`, docstring `:1102`; `app.py _harmonic_layer` verified in repo. `scripts/verify_track12_harmonic.py` exists and is read-only by reading (no DB, network or repo writes found). It checks only that harmonic functions do not mutate a synthetic audio buffer and a synthetic beat list (SHA-256 before and after). MEASURED It was run once for this memo and stopped on ModuleNotFoundError: librosa (exit code 1), so none of its assertions executed. unverified It never checks downbeat_times, real SANCT grids or cut points, so it cannot prove SANCT fidelity.	This is why the SANCT beat grid can stay untouched. The check script should be run before any release that changes this path.
Cost: `CREDIT_COSTS` for bpmandkey `quick_analysis` is base 5, cached 0. “quick” maps to `quick_analysis`. There is no separate harmonic charge line.	`webapp/app.py:30407-30411`; mapping at `app.py:30684` verified in repo	Harmonic data comes under the existing charge. Re-rendering stored data adds no compute and no charge. ESTIMATE (design reasoning).
Measured compute: BPM&Key totals 13.6 s CPU and 888 MB peak for a 229 s song (index.html:1027, from the strategy report), within that key detail 3.6 s, per-section key 2.6 s and chords 2.6 s (index.html:1027). A separate ~13.5 CPU-s figure appears at index.html:1032.	`index.html:1027`, `:1032` MEASURED (strategy report, via index.html)	Running the analysis is the expensive step. Re-rendering stored results is not.

Criterion	A: display-first	B: analysis-first	C: hybrid (thin elite display on a versioned additive JSON contract; analysis gated by phase 0 metrics)
User trust	2 of 5: a polished UI makes wrong labels look authoritative. ESTIMATE	4 of 5: labels appear only after they are checked. ESTIMATE	4 of 5: confidence drives rendering, so weak values look weak. ESTIMATE
Risk of shipping wrong labels	1 of 5: wrong labels ship on day one. ESTIMATE	4 of 5: little ships, so little can be wrong. ESTIMATE	4 of 5: only key, chords and confidence ship; section labels stay behind the gate. ESTIMATE
Rework cost	2 of 5: the UI is built around today’s 4 fixed windows. ESTIMATE	4 of 5: little is built yet, so little needs reworking. ESTIMATE	4 of 5: the contract absorbs engine change. ESTIMATE
Time to first visible value	5 of 5: fast, because `bpmandkey.js` already renders the data. ESTIMATE	1 of 5: nothing visible while phase 0 runs; the roadmap 6d step waits on datasets being sourced. ESTIMATE	4 of 5: bpmandkey first, then the player. ESTIMATE
SANCT beat-grid safety	4 of 5: display-only, but welded to the engine. ESTIMATE	4 of 5: the existing grid is documented as read-only (not proven), but engine iteration touches the analysis path. ESTIMATE	5 of 5: the proposed contract is a read-only annotation with no write path to `beat_times`. ESTIMATE
Compute and credits impact	4 of 5: no new analysis work. ESTIMATE	2 of 5: iterating analysis spends worker CPU. ESTIMATE	4 of 5: one analysis run per track; re-rendering is free. ESTIMATE
Demo value	5 of 5: looks finished. ESTIMATE	1 of 5: nothing to demo until phase 0 passes. ESTIMATE	4 of 5: a real demo of key, chords and confidence. ESTIMATE

Criterion	bpmandkey	SongChop player	Both at once
Surface area	scout-read Existing panels: key card _keyCard (bpmandkey.js:795), sections _sectionsPanel (:664), chord _chordPanel (:774). Surface is an analysis page.	verified in repo No existing reader of the harmonic fields (grep). Candidate hosts: stem-lane rows inside #stem-lanes-wrap (stem-lanes.js:76, :226-250), and the transport bar .cat-player-transport (catalog.js:1816). ESTIMATE Broader surface. (line numbers in bpmandkey.js / stem-lanes.js are scout-read)	ESTIMATE Sum of both surfaces, with two hosts to build and maintain.
Risk	ESTIMATE Low. Users opted in to analysis, and the result page is standalone.	ESTIMATE Higher. An extra analysis fetch is needed: the library list query does not select `ps.analysis` (app.py:15110). scout-reported Two waveform surfaces exist: stem-lanes.js:2-14 claims to be the only one, and catalog.js:1773-1825 still builds a WaveSurfer cat-player with regions. unverified which one is live.	ESTIMATE Highest. Doubles the surface before the contract has survived one engine change.
Reuse of existing chord timeline and key card	scout-read Highest. Panels already consume the JSONB (bpmandkey.js:494-495, :665, :777). The chord strip there ignores confidence.	ESTIMATE Low. Nothing reads the fields yet; the key card and chord logic must be ported or shared.	ESTIMATE Medium only if the renderer is shared. Otherwise the logic is duplicated.
Who sees it	ESTIMATE Users who opted in to analysis and open the result page.	ESTIMATE The main studio: a broader audience, including users who did not opt in.	ESTIMATE Both groups, so the first impression is set everywhere at once.
What it costs to get wrong	ESTIMATE Bounded. A wrong label sits on an analysis page and can be dismissed.	ESTIMATE Higher. A wrong label sits beside audio users trust to cut. Beat-grid cutting happens in the player, so the overlay must stay strictly read-only (AGENTS.md SANCT law).	ESTIMATE Compounded: two places where a wrong label can damage trust.
Verdict	First. Highest reuse, lowest risk.	Second, behind a localStorage flag (pattern: stem-lanes.js:104), once its live surface is confirmed and the contract has survived one engine change.	Not now. Revisit after the player step.

Evidence	Question asked of the candidate bar grid	Independent?	Tag
Segment lengths	Are section lengths whole 4, 8 or 16-bar multiples?	Yes, if lengths come from novelty, not the grid	HYPOTHESIS
Boundaries (novelty peaks, in seconds)	Do novelty peaks fall on this candidate's bar lines?	Yes, computed on a fixed hop grid in seconds	HYPOTHESIS
Chord changes	Do chord changes land on downbeats? Does the harmonic-rhythm period match bar length?	Only if chroma change is computed in seconds, not from SANCT beat times	VERIFIED IN REPO (circularity, see as-circular) HYPOTHESIS (scoring use)
Key changes	Do key changes fall at phrase starts? (weak, rare)	Yes, but low weight	HYPOTHESIS
Energy and layer entries	Do new instruments or loop layers enter at bar 1 of a phrase?	Yes, computed in seconds	HYPOTHESIS
Onset and low-frequency energy	Is onset and low-band energy higher on the downbeat than on the off-beat?	Yes. The existing vote is 0.6·onset + 0.4·low-band RMS	VERIFIED IN REPO (beat_engine.py:135-192, the vote) HYPOTHESIS (using it inside candidate scoring)
Tempo plausibility	Does the tempo this reading implies stay in a plausible range?	Yes	HYPOTHESIS (range TO BE TUNED, no numbers set)

Source	Finding	Number	Tag	Read depth	URL
Böck, Krebs, Widmer, ISMIR 2016	Explicit bar-length modelling matters where meter is not 4. Carnatic bars are the example.	Downbeat F1: Ballroom 0.863, Beatles 0.832. Carnatic 0.365 with 3/4 bars, vs 0.593 with bar lengths 3, 5, 7, 8.	MEASURED	Full text	Not recorded in our brief.
Durand, Bello, David, Richard, ICASSP 2015. Leave-one-dataset-out over 9 datasets. Each feature stream adds a significant gain. All six together beat any single feature by more than 18 points. The authors say the harmonic part is "a little less informative" on Ballroom. UNVERIFIED That is the authors' reading, not a controlled test. No chroma-only number was extracted from Fig. 3.	Mean downbeat F at 70 ms: Durand 67.5, vs 48.7 to 52.2 for three comparison systems. Beatles 82.2, vs 65.3 to 66.1. Ballroom 73.6, vs 45.2 to 50.3.	MEASURED	Full text	`https://perso.telecom-paristech.fr/grichard/Publications/2015-durand-icassp.pdf`
Böck and Davies, ISMIR 2020, Table 3 (8-fold CV). The authors say explicit-chroma systems beat learned-harmony systems on Beatles, because downbeats coincide with harmonic changes. They trail on rhythm-defined bars. This is a comparison between systems, not a controlled chroma ablation. UNVERIFIED as a controlled test. Table 2: a joint beat and downbeat DBN helps constant-meter music (Ballroom). It hurts expressive music (Hainsworth, SMC). The AMLt metric also accepts double or half tempo, triple or third tempo, and off-beat.	Downbeat F1. Ballroom: Böck 0.863, Durand (TASLP 2017) 0.797, Fuentes 0.83, theirs 0.916. Beatles: Böck 0.831, Durand 0.847, Fuentes 0.86, theirs 0.837. Hainsworth: 0.684 and 0.664 for the two comparison systems, theirs 0.722. GTZAN: 0.640 and 0.607 for the two comparison systems, theirs 0.672.	MEASURED	Full text	`https://program.ismir2020.net/static/final_papers/223.pdf`
Foscarin et al., Beat This!, ISMIR 2024. Drops the DBN postprocessing. The Sum Head cut downbeats more than 70 ms from a beat from 1.1% to 0.62% on GTZAN.	1.1% to 0.62% on GTZAN. Downbeat F1 tables did not render. UNVERIFIED A downbeat F1 of 78.3 on GTZAN is quoted in a secondary report (index.html). SCOUT-READ It is unchecked against the paper.	MEASURED	arXiv HTML, full read	arXiv ID not recorded in our brief.
Kim and Nam, 2023, all-in-one model. Joint beat, downbeat and structure training. The ablation is a bar chart with no extractable values. The direction is known. The size is not.	Directional only. No number.	UNVERIFIED	Full read	Not recorded in our brief.
Nieto et al., Harmonix Set, ISMIR 2019. 912 tracks with beats, downbeats and sections.	81.1% of segments start on a downbeat. Peak tempo is 128 BPM.	912 tracks. 81.1%. 128 BPM.	MEASURED	Full read	Not recorded in our brief.

Ambiguity	SANCT today	Structural signal that could disambiguate	Expected value ESTIMATE	Risk of wrong default applied	What the voting step does (sets editor default only if confidence and margin clear)
Downbeat off by 1 beat	Phase vote on onsets and low-band energy HYPOTHESIS	Chord changes and section edges on bar lines	Medium ESTIMATE	Medium	Scores the downbeat offset; may set editor start when margin is clear; else leaves today's default.
Downbeat off by 2 beats (half bar)	Can win if the backbeat is strong HYPOTHESIS	Harmonic rhythm and section edges at bar multiples	Low to medium ESTIMATE	Medium to high	Scores the reading; may set editor start when margin is clear; else leaves today's default.
Downbeat off by 3 beats	Same vote as above HYPOTHESIS	Weak. Chord changes on beat 3 can mislead.	Low ESTIMATE	High	Scores the reading; may set editor start when margin is clear; else leaves today's default.
Half-time or double-time tempo	Folds into 70 to 140 BPM SCOUT-READ `beat_engine.py` ~:482–502 (lines drift)	Chord-change period versus bar length; loop layering	Medium ESTIMATE	High. A tempo change moves every cut.	Scores the tempo reading; sets no editor default. Grid never changed.
3:2 hemiola reading	No special handling HYPOTHESIS	Chord changes spaced in threes; weak	Low ESTIMATE	Medium	Scores the reading; may set editor start when margin is clear; else leaves today's default.
6/8 versus 3/4 (or 12/8)	Assumes 4 beats per bar. Hard-code location not found in repo search. UNVERIFIED	Harmonic rhythm ratio; phrase length	Medium ESTIMATE	Medium	Scores the meter reading; sets no editor default. Beats per bar never changed.
Swing or shuffle (triplet feel)	No swing handling SCOUT-READ	Onset offsets inside the beat; no reliable signal found	Low ESTIMATE	Medium	Scores swing only; sets no editor default.
Off-beat emphasis (backbeat or off-beat accents)	Score is 0.6 times onset plus 0.4 times low-band RMS. Accents can win the vote. HYPOTHESIS	Chord changes and section edges	Medium ESTIMATE	Medium to high	Scores the phase; may set editor start when margin is clear; else leaves today's default.
Ratio 3:2 reading (editor preset)	No ratio choice; 4 beats per bar HYPOTHESIS	Chord-change period versus bar length	Low ESTIMATE	Medium	May set editor start to 3:2 when margin is clear; else today's default. Grid never changed.
Ratio 6:2 reading (editor preset)	No ratio choice HYPOTHESIS	Harmonic rhythm at 6-beat spans	Low ESTIMATE	Medium to high	May set editor start to 6:2 when margin is clear; else today's default. Grid never changed.
Ratio 2:1 reading (editor preset)	No ratio choice HYPOTHESIS	Chord-change period versus bar length	Medium ESTIMATE	Medium	May set editor start to 2:1 when margin is clear; else today's default. Grid never changed.
Off-beat lock (needs 2:1 + shift)	No lock to off-beat positions HYPOTHESIS	Loop layering on bar lines, with shift	Low ESTIMATE	High	May set editor start to 2:1 with a beat shift when margin is clear; else today's default. User can reset.

Item	Rule
Success	The share of songs whose initial editor settings already equal the final saved settings rises above today's default baseline. The threshold is TO BE SET BY THE FOUNDER. No number is proposed here. HYPOTHESIS
Golden diffs	Zero. HYPOTHESIS Plan target.
Hard stop	Any golden diff appears. Or the disagreement list is mostly false alarms. Or the voting step only ever agrees with today's default, so it gives no information. Then stop, archive, and write up. HYPOTHESIS
False-apply rate	Predicted default applied but the user changed it. Too high means stop. The rate threshold is TO BE SET BY THE FOUNDER. HYPOTHESIS

Field	Type or allowed values	Meaning
`ratio`	One of: none, 3:2, 6:2, 2:1, 4:1, 3:1, 1:2, 2:3	Winning ratio reading.
`shift`	Integer beat shift from the candidate set	Winning beat shift.
`downbeat_offset`	Integer, 0 to N-1	Winning downbeat offset.
`margin`	Number, 0 to 1	Gap between the winner and the runner-up.
`confidence`	Number, 0 to 1	Confidence in the winner.
`alternatives`	Top 3 readings with scores	Other likely readings.
`applied`	Boolean	True only when the editor opened with this reading (thresholds cleared, live mode).
`mode`	"shadow" or "live"	Shadow stores only. Live sets the editor's starting values.
`source`	"vote"	Fixed value.
`sanct_unchanged`	true	Fixed value. Asserts SANCT was not touched.

00Summary

Recommendations

How to read the numbers

01How our key detection works today

Why it is accurate (and where that is a claim, not a measurement)

Data we have to benchmark it

02Section / structure segmentation

Libraries and models

Realistic accuracy and the honest ceiling

What the numbers mean

By genre — what we can and cannot say

Commercial and competitor approaches

03Chord recognition & harmonic rhythm

How to stop beat-by-beat flicker

Harmonic-rhythm priors — what is and is not sourced

04Local / segmental key detection

What section structure adds to key (the core idea)

4b · Hierarchical inference is not the same as shorter windows

Hierarchical harmonic inference (what we propose)

Just shrinking the window (what we avoid)

05Beat / downbeat tracking as the grid

Why downbeats matter so much here

SANCT rule for this project

06Proposed architecture for ProduceHits

Bar-level chord decoding (sketch)

“Don’t over-segment” rules

6a · Recommendation table

6b · Ablations and benchmark plan

Ablations

Datasets and metrics

6c · Compute and licence notes

6d · Phased roadmap

07Honest risks

Label ambiguity

Electronic and loop-based music

Tempo changes and time signatures

Lyrics-driven structure

Over-segmentation and false modulations

Benchmark and licence risk

—Claim ledger (decision-driving claims, re-checked 2026-10-18)

—Sources

—Unverified list

Legend (same as the Research tab)

I0Verdict: HYBRID, own analyzer first

Recommendation

I1Evidence for the verdict

Why not “adopt everything”

I1b · Bounded issue-level evidence (root web_fetch, 2026-10-18)

Bounded mitigations

I2Sidecar design

I3Touch points: the ONLY places existing files would change

I4Repo laws check

I5Resource analysis

Current cost MEASURED (repo docs)

Incremental cost per song, by option

I6Prototype measurement MEASURED

What this does and does not show — read before quoting any number above

I7Product integration

I10Library-wide backfill

I8API response shape (design)

I9UI graph spec

I11Failure modes → mitigations

I12Phased roadmap

I13Sources, what is verified and what is not

I14Fluid re-analysis now has its own tab

F0Downbeat uncertainty, beat verification & inpainting: fluid re-analysis

Direct answer — five lines

F1 · Two layers, one grid revision, one cache key

F2 · Downbeat hypotheses with confidence — ask the human only where it matters

F3 · Variable meter, 2-beat interludes, pickups, half/double time

F4 · Edit types → what invalidates, and how far

F5 · Inpainting in this repo: grid metadata only, never audio

Verdict and one live trap

F6 · Decision table: reuse cache / Layer B only / recompute region / full re-process

Cached-feature storage per song ESTIMATE (from sample rate / hop / bins)

F7 · Trigger flow: a queued, idempotent, incremental job

F8 · Evaluation against human-verified grids

F9 · Sources and what is unverified

Our edge is a testable claim, not a result: bar-exact structure on our own grid, with honest confidence.

Five testable claims behind the plan — each with its test

Stage	What it adds	Gate that must pass	Current status
v1 alone	The frozen SANCT grid; exact `int(round(t*sr))` slices	Bit-identity vs the gold SANCT engine	docstring claim, NOT re-run in this work (acceptance requirement, unverified here)
v1 + v2 (proposed)	Sections, bar chords, per-section key as context feeding beat/downbeat inference; candidate output only	Synthetic chord-change regression	FAILED (`chords_distinct_in_path: 1`)
v1 + v2 + v3	Local drift/meter proposals as new, versioned revisions	Downbeat F (±70 ms) vs human-verified grids	NOT MEASURED
v3 without MIDI	Same as above, audio only	Same downbeat gate	NOT MEASURED
Downstream MIDI hypothesis (not v2/v3)	Section/key/chord context used to cut false/flickering notes while keeping genuine chromatic and fast notes	Note precision / recall / onset / offset; non-diatonic notes retained	NOT RUN

Version	What runs	CPU cost	GPU	Tag
v1	Shipped SANCT grid	0 new	none	shipped
v2 (proposed)	Structure/harmony-informed candidate engine	baseline ~13.6 CPU-s; v2 itself not measured	none	MEASURED existing harmonic-layer baseline, reused; v2 itself unmeasured
v3 (proposed)	Layer-B re-analysis	unmeasured	none	not measured benchmark required before any figure
v2 chord step (existing baseline)	Chord timeline	2.6 s	none	MEASURED (2026-10-09 strategy report, reused; existing baseline)
v2 per-section key (existing baseline)	Segmental key	2.6 s	none	MEASURED (existing baseline, reused)

Gate	Applies to	Pass condition	Current status
Bit-identity	v1	0.00 ms max diff vs gold SANCT	docstring claim, NOT re-run in this work (acceptance requirement, unverified here)
Synthetic chord-change regression	v2	Distinct chords preserved along the path	FAILED (`chords_distinct_in_path: 1`)
Benchmark vs rivals	v2	At or above the current chord/key baseline	NOT RUN
Downbeat F vs human-verified grids	v3	F clears its bar (±70 ms, `mir_eval`)	NOT MEASURED

Risk	Falsifier that would kill it
v2 may add no measured gain	Phase 0 ablation shows v2 at or below the current chord/key baseline.
The prototype decoder already failed	Already observed: `chords_distinct_in_path: 1`. A re-run must show > 1 distinct chord on the synthetic test.
v3 joint (phase, meter) Viterbi is our ESTIMATE	No primary source was opened for the joint-tempo-meter method itself; the downbeat gate either clears or it does not.
The golden pin is proposed as a tag, not a verified hash file	The `gold-sanct-beat` tag’s existence is unverified here; if it is missing or the gold engine is not reproducible, bit-identity cannot be re-proven.
MIDI context could suppress genuine chromatic or fast notes	If note precision/recall/onset/offset cannot be improved without deleting non-diatonic notes, the downstream MIDI hypothesis is dropped.
`analyzer_version` does not exist yet	It must be defined before it can key a cache; otherwise cache keys are incomplete.
Only Chromium was used for rendering	Cross-browser check (Firefox/Safari) surfaces layout or SVG differences.
No human-verified labelled set exists yet	The v3 gate cannot be measured until one exists; without it, v3 stays unmeasured.

Product	Key	Chords	Sections	Downbeats / bar	Granularity	Human correction	Runs where	Price / licence	Weakness	Label	Link
Bandcamp	No key field — the DDEX field list omits it verified	No	No	No	Metadata only — ISRC, title/subtitle, duration, artist, label, genre tags, synopsis, UPC/catalogue no., release year, P/C-line credits verified	n/a	Bandcamp storefront / platform	n/a (distribution platform)	Not an analysis competitor — metadata only	verified (DDEX field list) / unverified (help article list did not render)	bandcamp.com/ddex_guide · get.bandcamp.help
Moises	Key detection with pitch changer VENDOR CLAIM (page blocked to us)	Chord detection synced to playback; free tier first minute only VENDOR CLAIM (page blocked to us)	Automatic Sections, renamable on Premium VENDOR CLAIM (page blocked to us)	NOT YET RESEARCHED	NOT YET RESEARCHED	Sections renamable (Premium); chord editing NOT YET RESEARCHED	NOT YET RESEARCHED	Free tier (first minute of chords); Premium pricing NOT YET RESEARCHED	No accuracy figure published; help pages Cloudflare-blocked to us (page blocked to us)	unverified (page blocked to us)	help.moises.ai
rekordbox 7	Yes — “industry-leading key detection engine” VENDOR CLAIM	No chord-detection claim on the opened pages	Phrase Analysis: Intro / Up / Down / Chorus / Bridge / Verse / Outro / UNKNOWN + Fill in VENDOR CLAIM	BPM / beat grid VENDOR CLAIM; downbeat precision NOT YET RESEARCHED	Phrase-level	Phrases editable (cut / combine / change type / clear) VENDOR CLAIM	Desktop rekordbox	Paid product	No chord detection; phrase analysis is structural, not harmonic	VENDOR CLAIM	rekordbox.com · Phrase Edit guide (PDF)
Chordify	Key shown / transpose VENDOR CLAIM (page blocked to us)	Separate neural network for chords VENDOR CLAIM (page blocked to us)	NOT YET RESEARCHED	Separate beat network / tempo control VENDOR CLAIM (page blocked to us)	NOT YET RESEARCHED	NOT YET RESEARCHED	Web	NOT YET RESEARCHED	Technology page Cloudflare-blocked to us; no measured accuracy	unverified (page blocked to us)	chordify.net
Mixed In Key 11	Yes — claims results “at least 10% more accurate than any software that has ever existed” VENDOR CLAIM	No	Cue points (sections-ish) VENDOR CLAIM	NOT YET RESEARCHED	Key + Energy Level 1–10 + cue points	Cue points editable	Desktop app (macOS / Windows)	Paid	Accuracy claim unreplicated; key only, no chord or section analysis	VENDOR CLAIM	mixedinkey.com/learn-more
Serato DJ Pro	Key Analysis VENDOR CLAIM	No	No	Beat grid / Smart Sync VENDOR CLAIM; downbeat precision NOT YET RESEARCHED	Key + cue points	Cue points editable	macOS / Windows DJ app	Paid	Key only; no chord or section analysis	VENDOR CLAIM	serato.com/dj/pro
Traktor Pro 4	Key detection during track analysis; result in different notations and written to file tags unverified (manual surfaced via search, page not directly opened)	No	NOT YET RESEARCHED (stem separation advertised instead)	Beatgrid detection; hotcues VENDOR CLAIM	Key + beatgrid	Hotcues / beatgrid editing	macOS 13–15 / Windows 11 desktop	Paid product + Beatport streaming trial	No chord or section analysis advertised; stems only	VENDOR CLAIM	native-instruments.com/en/products/traktor
Ableton Live 12	NOT YET RESEARCHED	No audio chord-detection claim on the opened page	No	No analysis claim (manual warping) — NOT YET RESEARCHED	NOT YET RESEARCHED	Manual warping (n/a to analysis)	macOS / Windows DAW	Paid	No key / chord / section audio analysis for existing recordings on the opened page	VENDOR CLAIM	ableton.com/en/live
Logic Pro	NOT YET RESEARCHED	Chord ID / Chord Track VENDOR CLAIM	No	No	Chord	Chord Track editable	macOS	Paid	No section or downbeat analysis advertised; chords only	VENDOR CLAIM	apple.com/logic-pro
Cubase	Scale events (audio key detection NOT YET RESEARCHED)	Chord Assistant “Detected” tab — chords detected from audio events VENDOR CLAIM	Chord Track only; no section analysis	No	Chord events	Chord events editable; the audio should contain simultaneous chord tones unverified (search snippet; extraction page JS-blocked)	macOS / Windows DAW	Paid	Detection limited to audio events with simultaneous chord tones; no sections or downbeats	VENDOR CLAIM	steinberg.help (Detected tab)
Songsterr	No	Curated guitar / bass / drum tabs (community) VENDOR CLAIM	No	No (tab player synced to audio only)	Curated tabs	Community-edited tabs	Web + mobile app	Free + paid Plus	No audio analysis — curated tabs, chord shapes not detection	VENDOR CLAIM	songsterr.com/about
Hooktheory	NOT YET RESEARCHED	Crowdsourced song analyses (75k+) VENDOR CLAIM	NOT YET RESEARCHED	No	Curated theory analyses	Crowdsourced / curated	Web	Paid subscription (exact price NOT YET RESEARCHED)	Curated, not audio analysis	VENDOR CLAIM	hooktheory.com
SampLab Pro	BPM + key finder VENDOR CLAIM	“Identify harmonic changes and chord structures across an audio recording” — estimated chord labels with timestamps and structured JSON VENDOR CLAIM	NOT YET RESEARCHED	NOT YET RESEARCHED	Chord timestamps / lead sheet	Audio→MIDI editable	Browser tool (up to 50 MB, 1 s–10 min)	NOT YET RESEARCHED	The canonical samplab.com now serves Apple’s homepage and samplab.net is Cloudflare-blocked; only the .pro features page was opened	VENDOR CLAIM	samplab.pro/features
Captain Plugins	Set the project key to match the pitch of vocals / samples VENDOR CLAIM — composition/key selection, NOT confirmed audio key detection	Chord progression building VENDOR CLAIM — generation, not detection	Workspaces = verse / chorus structure VENDOR CLAIM	No	MIDI composition	MIDI import / export	VST / AU plugin	Paid	Composition / generation only; no confirmed audio key or chord detection	VENDOR CLAIM	mixedinkey.com/captain-plugins
Scaler	Scales / chord selection VENDOR CLAIM — NOT confirmed audio key detection	Chord progression generator VENDOR CLAIM — generation, not detection	No	No	MIDI composition	MIDI	Plugin	Paid	Composition / key-selection only; no confirmed audio key or chord detection	VENDOR CLAIM	scalermusic.com
Cyanite	No (tags are genre, mood, instruments, lyric theme, tempo)	No	No	No	Tags (genre / mood / instrument / tempo)	n/a	Cloud API	Paid API (exact price NOT YET RESEARCHED)	No key / chord / section / downbeat output advertised	VENDOR CLAIM	cyanite.ai
Melodyne	“Melodyne also recognizes chords, keys, tunings and tempos” VENDOR CLAIM	Yes — chord recognition (see key quote) VENDOR CLAIM	No	Tempo recognition, not downbeats	Note / chord note-level (DNA)	Full note editing (ARA / plugin / standalone)	macOS / Windows plugin (ARA / standalone)	Paid (5 editions)	Note / audio editing tool; no section segmentation or downbeat labelling	VENDOR CLAIM	celemony.com
Suno	NOT YET RESEARCHED (no analysis of uploaded audio claimed)	No	No	No	Generation	n/a	Web	Pro plan (500 songs/month); exact price NOT YET RESEARCHED	Stem separation into vocal + instrument only; no key / BPM / section analysis of uploads	VENDOR CLAIM	suno.com
Udio	NOT YET RESEARCHED	NOT YET RESEARCHED	NOT YET RESEARCHED	NOT YET RESEARCHED	NOT YET RESEARCHED	NOT YET RESEARCHED	NOT YET RESEARCHED	NOT YET RESEARCHED	Homepage served only a cookie-consent overlay; no product claims extractable	unverified (page served only a cookie overlay)	udio.com
LANDR	No	No	No	No	Mastering / stems / samples	n/a	Web	Studio from USD $8.25/mo VENDOR CLAIM	No key / chord / section detection claim	VENDOR CLAIM	landr.com
BandLab	No	No	No	No	Full DAW / mastering / loops	n/a	Browser / iOS / Android	Free	No key / chord / section audio-analysis claim	VENDOR CLAIM	bandlab.com
Soundslice	No	No (notation / tab editor)	No	No	Notation / tab editor synced to audio	Manual tab / notation editing (PDF / photo scanner)	Browser	Free tier + paid	No automatic key / chord audio detection	VENDOR CLAIM	soundslice.com
Ultimate Guitar	No	Curated chords / guitar tabs / bass tabs / ukulele chords catalog (community) VENDOR CLAIM	No	No	Curated tabs / chords catalog	Curated / community	Web	Free + Pro+	No audio-analysis claim	VENDOR CLAIM	ultimate-guitar.com
Spotify audio analysis	Audio Features (key, mode, tempo) — closed to new apps verified	No	Audio Analysis segments (not functional sections) verified	Beats / segments in Audio Analysis verified	Per-track features + segment-level analysis	n/a	Spotify Web API	Audio Features + Audio Analysis closed to new apps verified	Weakest link in this row: Audio Features and Audio Analysis are closed to new apps (official blog, 2024-11-27), so there is no free Spotify key / tempo / segment source for a new product verified	verified	developer.spotify.com blog
Tunebat	NOT YET RESEARCHED (page blocked to us)	NOT YET RESEARCHED (page blocked to us)	NOT YET RESEARCHED (page blocked to us)	NOT YET RESEARCHED (page blocked to us)	NOT YET RESEARCHED (page blocked to us)	NOT YET RESEARCHED (page blocked to us)	Web	NOT YET RESEARCHED (page blocked to us)	tunebat.com Cloudflare-blocked to us; no claim confirmed	unverified (page blocked to us)	tunebat.com
Splice	Library metadata — browse sounds by key VENDOR CLAIM (not analysis)	No	No	No	Sample-library metadata (genre / instrument / key / BPM)	n/a	Web + plugin	Subscription (exact price NOT YET RESEARCHED)	Key / BPM are library metadata, not analysis of user songs	VENDOR CLAIM	splice.com/sounds
all-in-one (allin1)	No	No	Functional segment labels (intro, verse, chorus, bridge, outro) VENDOR CLAIM	Beats + downbeats VENDOR CLAIM	Segment-level + beat / downbeat	None built-in	Python / GPU	Code MIT verified; checkpoint licence NOT stated (HF card 401 / empty) unverified	No key or chords; checkpoint weights licence unresolved	verified (code MIT) / unverified (checkpoint)	github.com/mir-aidj/all-in-one · HF checkpoint
SongFormer	No	No	Functional segment labels (intro / verse / chorus / bridge / inst / outro / pre-chorus / silence) VENDOR CLAIM	NOT YET RESEARCHED	Segment-level	None built-in	Python / GPU	Code CC-BY-4.0 verified; checkpoint + MuQ / MusicFM dependency terms unresolved unverified	Weights + dependency licences unresolved (issue #9 unanswered); MuQ reportedly CC-BY-NC	verified (code CC-BY-4.0) / unverified (checkpoint)	github.com/ASLP-lab/SongFormer · issue #9
BTC-ISMIR19	No	Chord recognition (bi-directional transformer) VENDOR CLAIM	No	No	Chord frames	None built-in	Python	MIT LICENSE file (Jonggwon Park, 2019) verified; training data is label-only	Unmaintained since 2020; chords only	verified (MIT)	github.com/jayg996/BTC-ISMIR19
ChordFormer	No	Chord recognition (Conformer) VENDOR CLAIM	No	No	Chord frames	None built-in	Python / GPU	Official repo `mwaseemrandhawa/ChordFormer` has NO LICENSE file (404); five fold checkpoints ship with no stated terms unverified; NOT `cameron-cs/chordformer` (an unrelated MIT project)	No stated code or weight terms anywhere in the official repo	unverified	github.com/mwaseemrandhawa/ChordFormer