The preset sweeps are one test per preset
The four sweeps that make a claim about a single preset — animation,
reactivity and sanity’s loudness gate — are generated, one #[test] per
shipped .toml, by the same core/build.rs glob that embeds the library. A
new preset gets its tests by existing; no Rust is edited. sanity’s shape gate
and distinctness generate one test per family instead, because their claims
are about a family’s distribution and its pairwise set and do not decompose
further.
Three things follow that matter when you are reading a red run:
- A failure names the preset, not a loop index inside a multi-minute test,
and
-E 'test(animation_attractor_ink)'re-runs exactly that one. - The per-preset reports print per preset.
sanity’s loudness ratio no longer arrives as one sorted table — sort a run’s lines to rebuild it — and the shape sweep’s flattest-preset ranking is scoped to the family whose test printed it. -P fastrenders a sample and the full run renders everything. A preset may declarerepresentative = true(see Parameter roster);build.rsmarks its generated tests<sweep>_rep_<stem>, and thefastprofile selects on that marker. So thedevlane’s per-phase gate renders 24 of the 81 presets, while a barecargo nextest run --workspace— what the plan close and CI’s coverage job run — renders all 81 plus the per-family shape and distinctness tests, which are never sampled. ADR-0081’s curation gate therefore still sees the whole library.
Individual tests (add -- --nocapture to see the printed diagnostics):
| test | kind | asserts |
|---|---|---|
reactivity | HARD | every preset moves for at least one band (bass/mid/treb/onset), driven by PCM through the real analyzer (Plan 0067 Phase 1 — see what the gates can and cannot see below); prints the per-band vector so a dead single binding — e.g. treble — is visible |
animation | HARD | every preset changes between frame N and N+k on at least one of two readings (not frozen). Since Plan 0123 Phase 1 (ADR-0136) the verdict is a disjunction over this gate’s own statistic: the silent reading below, or a driven one — the same footprint_diff between the silent capture at frame N+k and one taken against AnalysisFrame::fully_driven() at the same frame count, floored at 0.017 against the silent 0.01. A preset frozen in both still fails, and the static control is pinned failing both. Each preset’s own test prints which branch carried its pass, because that weakening is real and an unprinted property is one nobody re-reads. Since the sweep became one test per preset (ADR-0157) there is no end of a run to collect the still images in silence into one roster at; shot --presets presets --report’s anim column is where that set is read as a list now, and ADR-0136 carries the Outcome. Still no PCM: the driven capture is a synthesized frame, so the row in the stimulus table below is unchanged in kind. Scores metrics::footprint_diff since Plan 0077 Phase 1 (ADR-0091) — motion over the union of lit pixels, with bg_* stripped, so a sparse figure is measured against its own footprint rather than diluted by the frame. The rejected fifth-density emitter_squall draft passes at 0.1049 where the whole-frame statistic priced it out at 0.0057; a static control still fails on a zero numerator, and both are pinned as a standing non-vacuity test. Two things it still cannot do, both by construction rather than by tuning: a passing anim is not evidence of a watchable preset on a still family — an IFS figure is a photograph, so a slow view pan clears the floor while nothing in the figure moves (design-backlog 0066); and a rotationally symmetric figure cannot score its own spin — a figure invariant under rotation by 2*pi/k produces an identical image under that rotation, so its frame difference is zero at every resolution and no image-domain statistic lifts it (design-backlog 0009; the #[ignore]d resolution ladder from Plan 0067 Phase 1d is the recorded negative result, and it is why SIZE never moved). Such a figure must move radially, and that is an authoring constraint, not a gate defect |
sanity | HARD | every preset lights a minimum coverage and spans ≥2 quadrants (not blank, not a dot), and is not a blot. The blot check is a conjunction of two terms, and needs both to convict (ADR-0128, settled by ADR-0130, Plan 0119). Term one is tonal_flatness — the share of lit pixels inside one narrow luminance band, capped at MAX_TONAL_FLATNESS — asking does the figure have any tonal structure. Term two is metrics::boundary_density — the share of lit pixels touching an unlit 4-neighbour, perimeter over lit area, floored per system kind by boundary_floor — asking does the lit set have any interior. A preset fails only when it is flat and below its family’s floor; either alone convicts legitimate content, which is why neither is a verdict. Term one alone convicted a two-ink print for being two-ink (fragment_tiledmono, held out of the set for two plans over it); term two alone would convict 22 of the 43 shipped presets, which the gate prints as a count on every run. The tonal term is Plan 0056 Phase 5: a saturated single-tone mass satisfies coverage and spread completely, which is how a run of attractor presets shipped flat. What “lit” means moved twice, and neither reading is the preset’s own backdrop. ADR-0067 strips every bg_* binding for this capture, because a bg_vignette made the frame’s corner its darkest pixel and the backdrop read as a large, well-spread figure. ADR-0126 then made the reference the frame’s own ground — the mean tone of its most populous luminance band — rather than a hardcoded black, because a scene that paints its own paper reads full coverage whatever it drew, which made three of the four statistics constants for that content and left an emptying canvas indistinguishable from a broken one. So each statistic now answers how far does this picture depart from the ground it is drawn on: coverage how much of it does, quadrant_spread and radial_shell_occupancy where, tonal_flatness whether what departs has more than one tone, and boundary_density whether it has any interior. Most floors are measured from the library’s own distribution and printed on every run; boundary_floor’s default arm is not, and says so. It is 0.31, the midpoint of two frozen frames — the Blown Out blot at 0.2631 and fragment_tiledmono at 0.3602 — because a conjunction’s second term is judged only over the frames that failed the first, and that population has two members, one of which is the preset the term exists to admit. Half-the-sparsest would be circular there. The shape_collage arm (0.13) is the ordinary ceremony, on a family ADR-0123 holds under the tonemap knee so it has no additive path to the defect at all. KNOWN_FLAT is the exemption roster and it is empty — no shipped preset is excused from the blot check |
beat | HARD | a 120 BPM click track through the real DSP makes a beat-accent preset render differently on-beat vs off-beat; a zeroed beat binding does not |
distinctness | ADVISORY | prints per-family pixel + shape pairwise matrices and flags near-duplicate geometry; never asserts. Covers nine of the twelve shipped families, by a curated array in core/tests/distinctness.rs. The report’s unit is a pairwise matrix, so a family needs two or more presets before it can say anything — but that is no longer why the three absent ones are absent: shape_field (6), warp_mesh (4) and shape_collage (4) all ship enough and simply never joined the array, because a new SystemKind does not appear in it on its own and nothing fails when one is missing. Split per family since ADR-0157 — one test each, never sampled, each asserting its own pair count against the shipped set |
golden | HARD (tolerance) | one frozen fixture per system, plus the EXTRA_FIXTURES escape hatch below, matches its committed baseline PNG within a mean + max-outlier tolerance |
composite | HARD (tolerance) | the post stages, one fixture each and never all at once — trails, kaleido_*, bloom_*, plus one that binds no stage and guards the composite’s arithmetic (its assertion is that no channel of that fixture reaches 255 — a claim about the fixture, not a general property of the curve; see the re-bless note below). Captured at 160x100, a size whose internal grid is not the target’s shape, so an aspect error is visible |
bloom | HARD (relative) | the bloom stage’s behaviour, beside its baseline rather than in it: halo energy rises with bloom_amount, halo extent rises with bloom_radius, the rich tier’s deeper pyramid reaches further than the floor’s, and the halo is round. Captured at 256x256 — square, and load-bearing: the roundness guard is what catches a separable kernel whose two passes step in different units, and it reads 1.001 today against 7.05 under the defect it was written for. No magic numbers: every assertion compares two captures of one fixture differing in one bound param |
reaction_diffusion | HARD | the first stateful-feedback scene: seed reproducibility, regime response |
attractor | HARD | the first compute-particle scene: seed reproducibility + beat perturbation |
line_joints | HARD (+ tolerance) | a flagged joint stops leaving a hole in the stroke (ADR-0041): against a purpose-built zigzag polyline, a vertex is not a local luminance minimum relative to the segment interiors either side of it. Threshold-free, and captured at 512x512 because the wedge it measures is a fraction of a stroke-width across. The same capture is then pinned to a committed baseline (Plan 0040), since the reported defect had no pixel guard anywhere; the relative claim runs first, even under RLX_BLESS, so the notch cannot be blessed back in. Bless with --test line_joints, which cannot reach the golden roster |
attractor_trails | HARD (tolerance) | the attractor with the engine trails stage bound — the attractor’s four pipelines and the stage’s two in one command buffer, which is the densest pipeline coexistence any shipped preset produces and the thing ADR-0058’s hazard keys on. attractor.toml binds no trails and every composite_* fixture is a line scene, so nothing pinned it before Plan 0053. Captured at 160x100 (a non-square, per ADR-0037) and, like every baseline here, blessed on WARP — so it is coverage, not evidence of correctness; ADR-0058’s hardware-vs-WARP comparison is the check and this is the drift guard. Its own binary so RLX_BLESS=1 … --test attractor_trails can reach nothing else. A second, GPU-free test asserts the fixture still puts both accumulations live (trails > fade, spin non-zero), since at or below the scene’s own tail the stage is a bit-for-bit passthrough |
ink | HARD | the final tone-remap inverts tone, and ink_amount = 0 is byte-identical to an unbound frame |
geometry_extent | HARD | the in-frame geometry fraction, for the four line families only (ADR-0083): that the diagnostic is byte-identical to having it off, and that each of the two frozen over-scaled configurations measures below the shipped preset it was recovered from. Neither engine-wide nor a threshold — read the section below before using its numbers |
lit-backdrop guards (in-crate, --lib) | HARD (exact) | one per draw seam, three of them: swarm.rs’s a_lit_backdrop_survives_where_the_swarm_drew_nothing, lines/renderer.rs’s a_lit_backdrop_survives_where_the_strokes_drew_nothing, and emitter.rs’s a_lit_backdrop_survives_where_the_emitter_drew_nothing (ADR-0056). Each captures swarm_lit_backdrop.toml / lines_lit_backdrop.toml / emitter_lit_backdrop.toml three ways — lit backdrop, black backdrop, backdrop with the scene contributing nothing — and asserts that wherever the scene wrote no light the backdrop arrives intact. Bound 0 rather than a tolerance, because it reads the linear composite; see the section below. The swarm’s and the lines’ take a fourth capture at zero emitted light (Plan 0053 Phase 4), which turns the frame into a direct readout of alpha and widens the line guard’s reach from 15 channels to the whole stroke footprint |
emitter burst (in-crate, --lib) | HARD (relative) | the emitter is the first scene whose population varies, so emitter.rs’s a_spawn_rate_on_onset_bursts_and_then_idles drives emitter_onset.toml through capture_preset_over with a silent lead, a six-frame transient and a second of silence, and asserts the frame is dark before, lit after, and dark again by the end. capture_preset cannot ask this: it holds one analysis frame for every step, so it can show that a binding is live but never that the shower empties when the transient passes |
background_composite | HARD (hardware only) | RD / attractor presents alpha-blend over the bg_* backdrop; skipped on a software adapter, which mis-renders that pipeline set. The stated cause of that mis-render was identified and fixed by Plan 0053 Phase 3 — it was background-bind-layout colliding with rd-init-layout / fragment-field-uniform-layout (ADR-0058), and an explicit min_binding_size moved WARP onto the hardware numbers for the RD half (087.612 165.165 156.168 hardware, against a bare layout’s 087.543 064.538 …). Whether the skip can now be lifted is open and unmeasured: the attractor half of this test is a different layout group and nothing probed it. The module docs here and in background_composite.rs still assert the quirk as live — do not read that as evidence it is, and do not lift the gate without rendering both halves on both adapters |
transition | HARD | every switch path (cycle and select) renders intermediate blended frames as a ramp, reproducibly from the injected dt; each blend kind shows its own signature; a switch arriving mid-dissolve lands on the last index requested; a hot-reload mid-dissolve cancels cleanly; the heavy attractor ↔ reaction-diffusion pair dissolves on the freeze fallback (set RLX_TRANSITION_STRIP=<dir> to also dump filmstrips) |
easing | HARD | [smoothing] is observable: a scalar entry measures symmetric and an { attack, release } pair does not, against purpose-built near-linear fixtures (ADR-0039). Also measures the spectrum curve↔easing ordering both ways round through one renderer — every frame count in the suite is gated on segment_settled first — the shared probe’s window is 180 frames (3 s, 6 τ) because at 96 its own asymmetric arm was truncated, reading 61 where the settled answer is 69 |
preset | HARD | the expression evaluator and TOML schema: exact values, rejection without panic, zero allocation per eval, and the PARAMS ↔ set_param drift guard |
dsp / ffi / hygiene | HARD | known-signal analysis fixtures; the C ABI across the boundary; the hot-path panic pragma + exact dependency pinning |
Golden baselines pin frozen fixtures, not shipped presets. core/tests/fixtures/*.toml
is a deliberately minimal preset per SystemKind, committed alongside
core/tests/golden/*.png; the shipped presets in presets/ are guarded
behaviorally (sanity / reactivity / animation) so the preset-author lane can
tune them freely without re-blessing pixels. A new SystemKind variant fails
golden.rs to compile until its fixture exists (exhaustive match, no wildcard arm).
One narrow exception: EXTRA_FIXTURES (Plan 0063). A second fixture for a system
already in the roster, for when the rostered one structurally cannot reach the code under
test. attractor_depth.toml opened it, and the list has grown since: the rostered
attractor.toml is De Jong, and ADR-0076
gives every 2-D family an inverse depth extent of exactly 0.0 — which is precisely what
makes the perspective divide, the distance haze and the depth tint the identity there, so
no edit to that fixture could execute a line of them. The newest, warp_mesh_shader.toml
(Plan 0110), earns its place the same way one level up: it is the only fixture anywhere in
the crate that carries WGSL, and render/scenes/warp_mesh/shader.rs is built only for a
bundle that declares a shader — so no edit to the rostered warp_mesh.toml, or to the
bytecode-driven warp_mesh_milk.toml beside it, could execute a line of that file either.
The list is captured after the
roster loop, never interleaved with it, so every pre-existing baseline renders from the
device state it always did (which matters on WARP, where building GPU resources mid-run
changes what a later capture resolves to). systems_rosters_every_variant holds it to the
roster’s own two conditions plus one of its own: a stem colliding with a rostered system’s
would have the two silently overwrite each other’s baseline. The roster stays exhaustive —
ADR-0023 rests on that and this does not weaken it.
core::signal (pure, zero-dep) synthesizes the test audio; core::render::metrics
(pure) provides frame_diff, struct_diff, coverage, and quadrant_spread,
shared by the tests and the CLI report, plus the step-response pair
frames_to_settle / step_response and the segment_settled gate that says
whether either of those two is worth reading.
Built from a8ce055 at version 0.115.0. This site tracks main and is not versioned per release.