Files
effectcraft/docs/testing.md
bflatasticandClaude Opus 5.5 e61f0e5560 Effects: the sim golden test accepts macOS 27's libm hash for Mr. Mercury (#232)
On macOS 27.2 (rustc 1.98) one of the 404 pinned simulation hashes,
CC Mr. Mercury at non-default settings, 32 bpc, t = 2.2 s, came out
different. The reporter rendered every case at the commit that pinned
the hashes and at current main on that machine, and the frames were
byte-identical. So the code didn't change: the platform's libm did
(the hashes are exact f32 bits through sin, cos and powf). The test
now also accepts listed per-case libm variants, with that hash as the
first, and every other case stays exact. Its docs say how to tell a
libm difference from a code change.

Test: libm_variants_are_accepted_for_their_case_only (sim_golden; runs
on every platform).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:36:09 +07:00

2.3 KiB

Testing

cargo test --workspace     # everything
cargo xtask ci             # fmt, clippy -D warnings, tests, layers, assets, wasm

What the tests cover

  • Engine commands (crates/engine/src/tests*.rs): every command family, including undo and redo, and that every menu entry resolves to a registered command.
  • Rendering (crates/render): pixel checks on small compositions, for blend modes, masks, mattes, time remapping, 3D projection, lighting and shadows. Fast paths (the layer cache, the affine compositor, blurs, motion blur accumulation) are checked against their reference implementations.
  • Effects (crates/effects): each effect has a determinism test and at least one behaviour test (identity at neutral settings, known pixel results, or simulations giving the same frame whether you seek straight to it or play up to it). A registry-wide test checks that effects which read the clock are declared time-dependent. crates/effects/tests/sim_golden.rs pins exact pixel hashes of the simulation effects the GPU shares plans with (CC Rainfall … Card Wipe) at several settings, times, bit depths and resolutions, so any change to their CPU output fails (re-pin with SIM_GOLDEN_PRINT=1 after an intended change; pinned for aarch64 macOS, with the hashes another macOS release's libm gives for unchanged code in LIBM_VARIANTS).
  • Export (crates/export/tests): every format is encoded and decoded back, checking frame count, size and pixels. When ffmpeg/ffprobe are installed they are used as an outside check; they are never linked or shipped.
  • Automation (crates/automation, apps/effectcraft-cli/tests): MCP protocol round trips and the command-line tool's JSON output.
  • Interface (crates/ui-egui): headless egui_kittest tests for panels and dialogs.

Looking at the interface without a window

cargo run -p effectcraft-ui-egui --example snapshot -- --out ui.png \
  --step '{"method":"engine.execute","params":{"command":"layer.select","params":{"layers":["#2"]}}}'

Each --step is a control-channel request; {"method":"snap","params":{"path":"x.png"}} writes an intermediate image.

Benchmarks

cargo run --release -p effectcraft-cli -- bench --n 10 --play 30

prints per-layer and per-effect timings for one frame, and playback timings with and without the layer cache.