From ebff7caaaf348602422f6943e9de01240a79994e Mon Sep 17 00:00:00 2001 From: blasty Date: Thu, 6 Aug 2026 22:05:03 +0200 Subject: tests: one front door Fourteen test files, each its own __main__, and no way to run them but from memory -- so in practice you ran the one you were working on and hoped. Worse, nothing said which files need a licensed IDA and a real worker (minutes) and which are pure stdlib (milliseconds), so the cheap ones nobody ran either. tests/run.py runs the lot and prints one table. --fast selects only the suites that need nothing, which is 257 checks in half a second under any python3 -- that's the one you run between edits. The classification lives in the test files, not in a table here that would rot the first time someone adds a test: each declares NEEDS_IDA at module scope and run.py reads it with ast (it can't import them -- they run their suite at import). A file without the marker is a hard error rather than a silent guess. --- README.md | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index e3710d9..d857989 100644 --- a/README.md +++ b/README.md @@ -195,12 +195,29 @@ See `docs/RPC.md` for the full protocol. ## Tests -A headless Textual `Pilot` suite lives in `tests/`; it spawns a worker on the -given binary (default `targets/echo`): +`tests/run.py` is the front door — it runs every suite and prints one table: ```sh -python tests/test_scenarios.py targets/echo # full UI suite -python tests/test_scenarios.py --only hex,rename +python3 tests/run.py # everything (needs IDA; ~3 min) +python3 tests/run.py --fast # only the no-IDA suites — ~0.5s, runs anywhere +python3 tests/run.py --list # what would run, and whether it needs IDA +python3 tests/run.py trace -x # only files matching "trace", stop at first failure +``` + +Test files come in two kinds and **each one declares which** with a module-level +`NEEDS_IDA` marker (`run.py` reads it without importing the file, and refuses to +run if a file doesn't have one): + +- **pure** — stdlib only, no IDA, no worker, no binary. Seconds. This is what + you run between edits. +- **IDA** — spawns a real idalib worker on a real target and drives the headless + Textual `Pilot` against it. Minutes, and needs a licensed IDA. + +The individual suites still run standalone, which is how you iterate on one: + +```sh +~/ida-venv/bin/python tests/test_scenarios.py targets/echo # full UI suite +~/ida-venv/bin/python tests/test_scenarios.py --only hex,rename ``` ## Docs -- cgit v1.3.1-sl0p