diff options
| author | user <user@clank> | 2026-08-07 09:13:33 +0200 |
|---|---|---|
| committer | user <user@clank> | 2026-08-07 09:13:33 +0200 |
| commit | 0fc0b3a9d667f7f464939bce8bfdf4cd9cd889bb (patch) | |
| tree | f8c30d42e79b683b2f7770f2caeecbedcaab22ae /.auto/ideas.md | |
| parent | decomp_map: memoise obj_id -> ea for the whole function instead of only compa... (diff) | |
| download | ida-tui-0fc0b3a9d667f7f464939bce8bfdf4cd9cd889bb.tar.gz ida-tui-0fc0b3a9d667f7f464939bce8bfdf4cd9cd889bb.tar.xz ida-tui-0fc0b3a9d667f7f464939bce8bfdf4cd9cd889bb.zip | |
Park two proven-but-unresolvable F5-path optimisations, and record how to spot a counter-drift outlier
pc_nums allocated three ctree_item_t SWIG objects per candidate column -- the
third instance of the same fault already fixed in decomp_map and found in
decompile_function_safe. 1247 -> 614ms warm over 18991 lines of bash, identical
literal counts, 0 mismatches over echo's 128 functions.
Both it and the fast decompile_function_safe are parked rather than applied:
together they are worth ~350ms against a run-to-run spread of ~500ms on this
box (means 25045 with, 25140 without over seven runs), so the benchmark cannot
resolve them. Neither adds complexity -- both remove allocations -- so they are
kept on disk with their measurements for a per-operation-latency goal.
Also records the 27283ms outlier: the work counters moved with it, which is how
an outlier is told from a regression.
Diffstat (limited to '.auto/ideas.md')
| -rw-r--r-- | .auto/ideas.md | 99 |
1 files changed, 99 insertions, 0 deletions
diff --git a/.auto/ideas.md b/.auto/ideas.md index 9586574..133993c 100644 --- a/.auto/ideas.md +++ b/.auto/ideas.md @@ -230,3 +230,102 @@ come from the same ctree. `Program.decompile` calls the `force_recompile` tool `ida_hexrays.decompile()` repopulates the cache and `decomp_map` then hits it. `/tmp/mapalign.py` appeared to show a mismatch only because the probe itself used `DECOMP_NO_CACHE`, which the app never does. + +## PARKED: a 3.6x faster decompile_function_safe (measured, byte-identical, discarded on the metric) + +ida-pro-mcp's `decompile_function_safe` — the function that produces the +pseudocode the pane shows — has the **same three faults** that were fixed in +`decomp_map`: + +* it allocates **three** `ctree_item_t` SWIG objects per pseudocode line, and + `_head` and `_tail` are never read (`get_line_item` takes None for both); +* it calls `dstr()` per line to recover the `/*0xEA*/` marker — 24 µs a call — + where consecutive lines of a multi-line expression report the same ctree item, + so memoising by `obj_id` (unique within a cfunc) skips most of them. + +Measured, on a **warm** cfunc (so this is pure post-processing, no Hex-Rays): + +| workload | before | after | +|---|---|---| +| bash's 8 largest, 18 991 lines (`/tmp/decprof.py`) | 2 302 ms (121.2 µs/line) | 429 ms (22.6 µs/line) | +| the same through the real worker (`/tmp/verifybind.py`) | 2 695 ms | 746 ms | +| echo's 12 largest, 3 298 lines (`/tmp/splitprof.py`) | 219 ms | 120 ms (30 µs/line) | + +Byte-identical: `check_decomp.py` runs both implementations against the same +cfunc with `include_addresses` both ways — **128/128 echo and 372/372 ls_ttl**. +It is a real gate: keying the memo on `it.op` instead of `it.obj_id` fails 69 of +128. + +**Discarded anyway**, because it does not move `total_ms`. Three runs with it +(25 367 / 25 058 / 25 171) against three without (24 514 / 25 122 / 25 783) — +the means are 25 199 vs 25 140, i.e. indistinguishable. The reason is in the +third row of the table: the saving is 98 µs/line on bash's *largest* functions +but only 30 µs/line on small ones, and the bench's fixed set averages 286 lines +a function. Expected effect ~150–250 ms against a run-to-run spread of ±400–600 +on this box. + +**It is still a real win for the operation a user waits on** — an F5 on a +2 374-line function drops 287 ms → 54 ms of post-processing — so it is parked +rather than deleted: + +* `.auto/parked/fast_decompile.patch` (applies to `server/patch_server.py` and + `idatui/worker.py`) +* `.auto/parked/check_decomp.py` (re-wire into `checks.sh` if the patch is + re-applied; it will crash if run without it, since it slices the function out + of `BODY`) + +Re-apply it if the benchmark ever decompiles large functions, or if the goal +moves from total session time to per-operation latency. Do **not** re-shape the +bench's fixture set to make this win visible — that would be fitting the +benchmark to the change. + +## PARKED: pc_nums, the third instance of the same bug (2.03x, byte-identical) + +`_idatui_pc_nums` allocated **three `ctree_item_t` SWIG objects per candidate +column** — inside a scan that probes every literal-looking character of every +pseudocode line. `a`–`f` are hex digits, so `a1`, `v6` and `sub_1F4C0` all +qualify and most columns of a line get probed. `head` and `tail` were never +read. This is the same fault as `decomp_map`'s sweep and +`decompile_function_safe`'s loop — **three instances of one bug**. + +It is on the F5 path: `app.py:_load_decomp` fetches `pc_nums` after every +successful decompile so the view can mark the literal under the cursor without a +round trip per keypress. + +Measured warm (Hex-Rays already cached), bash's 8 largest, 18 991 lines: +**1 247 ms → 614 ms (2.03x), 66 → 32 µs/line**, with the literal count identical +(7 015). Also stopped `tag_remove` running twice over every line (`pc_nums` and +`_idatui_pc_nums` each called it); worth ~nothing on its own but it is strictly +less work. + +Equivalence: the tool's whole output dumped per function and compared across +revisions (`/tmp/pcnumdump.py`, cross-process — an in-process differential +**segfaults**, two SWIG item objects over one cfunc). echo: 128 functions, +1 463 literals, **0 mismatches**. `tests/test_scenarios.py` also drives literal +cycling through these exact column extents. + +Parked at `.auto/parked/fast_pc_nums.patch` for the same reason as the decompile +patch: real work removed, but ~170 ms against a ±500 ms run-to-run spread. + +## Both parked patches, measured together + +Applied together (they are both on the F5 path) over four runs: 25 367 / 25 058 / +25 171 / 24 585, against three without: 24 514 / 25 122 / 25 783. Means **25 045 +with, 25 140 without** — a 95 ms edge inside a 500 ms spread, i.e. still not +resolvable. Discarded on the metric, kept on disk. + +Apply both if the goal moves to per-operation latency: an F5 on a 2 374-line +function loses ~233 ms of text post-processing and halves its `pc_nums` cost. + +## A 27 283 ms outlier, and how to recognise one + +One run came back at 27 283 (against ~24 500) with `lg_split` 2 200 → 6 917 and +`lg_search` 3 441 → 840. It was NOT the change under test: the **work counters +moved with it** — `lg_decomp_lines` 3 436 → 3 233, `lg_split_mapped_lines` +2 070 → 1 993, `lg_search_hits` 91 783 → 92 733. Hex-Rays decompiled bash +differently that run (the drift documented above), which changed how much of the +post-rename re-render the split phase absorbed before search got to it. The next +run reproduced 24 585 with every counter back to its usual value. + +**The NOTES counters are what tell an outlier from a regression.** A real +regression moves the time and leaves the work alone. |
