ai: Add testing guidance for tools/

This commit is contained in:
Marshall Greenblatt committed 2026-07-27 14:06:00 -04:00
1 parent d43e90df57
commit 65e50f1c0b
2 files changed
+112

No files matched your search

+111
View File
@@ -0,0 +1,111 @@
# CEF Tools Contributor and Agent Guidelines
This file applies to `//cef/tools` and its descendants.
## Generator inventory and coverage
- Every direct-child, non-test `make_*.py` generator must have a matching
`make_*_test.py` module.
- `make_generator_inventory_test.py` enforces the direct-child inventory. When
adding or removing a generator, update its expected count intentionally and
keep nested generators, helpers, and existing tests out of the inventory.
- Exercise every top-level generator function through at least one successful
test. Cover meaningful explicit error branches, writer behavior, and the
generator's CLI style; do not treat filename coverage as function coverage.
- Use standard-library `unittest`, `tempfile`, and `unittest.mock`. Run from the
CEF root:
```sh
python3 -m unittest discover -s tools -p '*_test.py'
python3 -m unittest discover -s tools -p 'make_example_test.py' -v
```
- Run the complete suite twice for generator-wide work. These modules share
parser state, so the second run detects ordering and cleanup leaks.
## Hermetic fixtures and goldens
- Put deterministic inputs and reviewed goldens under
`tools/testdata/make_generators/`; use `generator_test_util.py` for shared
fixture, manifest, archive, and CLI helpers.
- Mock or fix time, year, Git metadata, versions, platforms, environment
variables, external commands, and formatter results when they affect output.
- Goldens must originate from a reviewed baseline, never from ignored
working-tree output, and tests must not contain an auto-update path.
- Use inline expectations for small output and reviewed full-file goldens for
larger generated C/C++ content. Add targeted assertions that explain the
behavior protected by a golden.
- Prove that customized CppToC/CToCpp bodies survive regeneration, source
fixtures are not modified, and relevant paths work when they contain spaces.
## Writer contracts
- Test every writer on its first generation and an identical second
generation.
- For `write_file_if_changed`, assert changed/unchanged returns and preserved
mtime. For unconditional writers, assert stable bytes across rewrites.
- Multi-output writers must report a change when any output changes. Test
repairs where only one output was corrupted.
- Pre-publication failures must leave outputs missing or unchanged. Characterize
partial I/O-failure output before changing it; atomic publication is a
separate behavior change.
## CLI contracts
- Cover invalid status, stdout, stderr, and partial artifacts, plus successful
stdout, single-file, multi-output, argparse, and distribution CLI styles.
- If a generator function returns metadata with its content, such as
`(content, customized)`, unpack the result at the CLI boundary and write only
the string content. Do not change the underlying generator return contract
merely to fix a CLI.
- Assert progress messages exactly when callers may depend on them, and return
explicit nonzero statuses for rejected invocations.
## Parser and module state
- Restore parser notification context and other module globals with
`try/finally`; add test cleanup so failing regressions cannot leak state.
- Generator modules intended for reuse must be importable without parsing
arguments, exiting, mutating files, calling Git, or launching subprocesses.
Put orchestration in `main(argv=None)` and use a conventional guarded entry
point when refactoring for importability.
## Output equivalence
- Establish an exact pre-refactor baseline and keep testability refactors
separate from behavior fixes.
- Testability refactors must preserve generated bytes, paths, ordering,
comments, whitespace, guards, encoding, newlines, modes, and symlink targets.
- Compare fixes to the latest approved baseline while retaining the original
refactor baseline as an immutable reference.
- Return-value, status, diagnostic, and CLI changes must not alter generated
files unless that is their documented purpose.
- For translator changes, run a repository-scale comparison covering C API,
CppToC, CToCpp, wrapper, dylib, and GYPI outputs.
- For distribution changes, compare the complete normalized tree. Compare
archive member paths, types, uncompressed bytes, and modes; normalize archive
timestamps only. Do not use raw archive-container hashes as equivalence.
- Use the same Python interpreter as the recorded baseline. Python-version
differences can change comment wrapping and produce misleading byte deltas.
## Bug-fix workflow
- Reproduce a bug with a focused regression before changing production code.
- Keep coverage/refactor checkpoints separate from behavior fixes so output
differences cannot be attributed to the wrong change.
- Keep unrelated bugs independently reviewable. Closely related instances of
the same root cause may be grouped when explicitly requested.
## Formatting and validation
- From the CEF root, format only explicit changed Python paths:
```sh
python3 tools/fix_style.py tools/make_example.py tools/make_example_test.py
```
- Inspect formatter output immediately. The formatter may rewrap unrelated
legacy code; revert that churn with targeted patches.
- Compile every changed generator, helper, and test.
- Run `git diff --check` and scan untracked files for trailing whitespace.
- Re-run focused tests after formatting, then run the complete suite twice.
+1
View File
@@ -0,0 +1 @@
@AGENTS.md