From 21971d0a9ce9b8a2fb1db2ec67303696cea9cbbb Mon Sep 17 00:00:00 2001 From: blasty Date: Fri, 7 Aug 2026 14:32:56 +0200 Subject: tests: blob_ui 39.8s -> 4.1s, and a fatal it was hiding Three separate wastes, all of the same family: waiting on a guess instead of a signal, and paying for work that never had to be repeated. 1. The 64KB blob was built with os.urandom into a fresh TemporaryDirectory on every run. New bytes at a new path means the pristine-database cache can never apply, so full auto-analysis of 64KB of AArch64-decoded noise was paid every single run. It is now built from a seeded PRNG at a stable path (tests/.synthetic/, gitignored) and staged through the existing cache. Determinism is also a correctness fix: whether 64KB of chance bytes contains something IDA reads as a function is luck, and this suite asserts "and really has no functions". 2. `wait(lambda: lst.model is not old, ..., 30)` after commenting. The perf work made an item edit KEEP the listing's walk and re-render in place, so the model object is never replaced and this waited out its full 30s timeout on every run -- and then "commenting leaves the view where it was" passed vacuously, because nothing had happened at all. A test that burns 30s to check nothing is worse than no test. 3. Two `pause(2.0)`/`pause(2.5)` after a carve, replaced with settle() on a real condition. The second one deliberately has NO predicate: that spot is random data, so the carve may legitimately produce nothing, and "the row became code" would never hold -- gating on it cost another 30s timeout. What that check is about is the VIEW not moving, so the gate is "the app finished reacting". Fixing (1) exposed a real bug in the client, fixed here too: reopening a database that already exists while passing loader switches is FATAL in IDA -- FATAL ERROR: Switch '-b400' can be used only when loading a new file which kills the worker before it can report anything. Loader switches describe an IMPORT and are recorded in the database they produce, so they are now sent only when there is an import to describe. This was never reachable from the old suite (a fresh random blob never had a database to reopen), but it is reachable by any user who opens a raw blob with --ida-args twice. 30 passed, 0 failed. --- tests/_fixtures.py | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) (limited to 'tests/_fixtures.py') diff --git a/tests/_fixtures.py b/tests/_fixtures.py index c7a45d3..0b72856 100644 --- a/tests/_fixtures.py +++ b/tests/_fixtures.py @@ -37,6 +37,36 @@ def cache_is_fresh(binary: str) -> bool: return os.path.exists(c) and os.path.getmtime(c) >= os.path.getmtime(binary) +#: Generated targets live here so their pristine caches survive between runs. +#: Gitignored; safe to delete (the next run rebuilds both). +SYNTHETIC_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), ".synthetic") + + +def synthetic(name: str, build) -> str: + """A generated binary at a STABLE path, rebuilt only when its bytes change. + + Generated targets used to be written into a fresh TemporaryDirectory on + every run, which quietly defeated the whole pristine-cache scheme: a new + path with new bytes every time means auto-analysis is paid in full, every + run, forever. `test_blob_ui`'s 64KB blob cost ~40s a run that way. + + ``build()`` must be DETERMINISTIC and return bytes. That is also what makes + the suites reproducible: a blob built from os.urandom can, by luck, contain + something IDA reads as a function, and then a test asserting "no functions" + fails for reasons no one can reproduce. + """ + os.makedirs(SYNTHETIC_DIR, exist_ok=True) + path = os.path.join(SYNTHETIC_DIR, name) + data = build() + if not os.path.exists(path) or open(path, "rb").read() != data: + with open(path, "wb") as fh: # content changed -> cache is stale + fh.write(data) + for stale in (cache_path(path), path + ".i64"): + if os.path.exists(stale): + os.remove(stale) + return path + + async def build_pristine(binary: str, cache: str, app_factory) -> None: """Analyse ``binary`` once and keep the database as a golden copy. -- cgit v1.3.1-sl0p