path: root/.auto/ideas.md

# Ideas backlog

## Perf, not yet tried

- **Skeleton walk for `ensure_ea`** — navigation only needs the *row index* of an
  address, yet `ListingModel` walks the segment loading fully-rendered rows.
  **Costed, and it only nets ~5%**: a text-free walk would be ~3 µs/row instead
  of ~23, but search then has to fetch the text anyway (it builds its haystack
  from `_line_plain`), so most of the saving moves rather than disappears.
  Worth it only if the goal changes from "total session time" to "no single
  foreground wait over 200 ms" — which is arguably the better goal for a TUI.
- **`_grow` should start from where the user is**, not sweep from the segment
  start, so a jump into the middle doesn't wait behind everything before it.
- **Prefetch the decompilation of adjacent/called functions** while the user
  reads the current one. The worker is idle then and `Program.decompile` caches.
- **Persist the worker's `_idatui_line_parts` cache** — it is warm only within
  one worker, and the same binary is reopened constantly during a session.

## Bugs found while optimising (not perf work)

- **`idatui/graph.py` edge routing is non-deterministic.** Laying out the *same*
  function twice with the *unchanged* engine gives different
  `painting.vruns` for 71 of 128 corpus functions. Node placement is stable;
  only the routing moves. So a graph redraws differently when you reopen it, and
  any old-vs-new painting diff is worthless as a regression test (old-vs-old
  fails it too). Find the set/dict iteration or `id()`-keyed order behind it.
- **Sticky graph mode makes a keypress ambiguous.** With `_graph_sticky` on, a
  navigation schedules the next function's graph asynchronously; until it lands
  the app is in the listing. So `space` right after a jump either enters or
  leaves the graph depending on which won. `graph_minimap` was silently relying
  on losing that race. A fix would be to enter graph mode immediately, with a
  loading state, when a sticky navigation starts.
- **`domain.decomp_map` costs ~280 ms per function** — more than the decompile
  itself — and is on the split-view (`s`) path, which the bench doesn't cover.

## Verification patterns that worked (reuse them)

- **Differential against a git ref.** `.auto/diff_spans.py` loads the *current*
  and a *past* `server/patch_server.py`, execs the same slice of each `BODY`,
  and compares outputs over every disassembly line of a real binary. Catch:
  `BODY` is a normal triple-quoted string, so you must import the module and
  read `mod.BODY` — slicing the file text tests an undecoded program.
- **Compare what reaches the screen, not the data structure.** Merging Segments
  is *supposed* to change the segments; expand both to `(char, style)` per cell
  and compare that (`/tmp/hexeq.py`, `/tmp/gveq.py`).
- **Check the old code against itself first.** The graph routing diff looked
  like a regression until old-vs-old failed identically.
- **`.auto/check_search.py`** is the permanent version of this idea and runs in
  `checks.sh`: it compares the search fast paths against the plain per-line loop
  for every typed prefix. Cache staleness returns a *plausible wrong answer*,
  which no scenario test can catch.
- The bench's `NOTES` counters (`search_hits`, `graph_blocks`, `decomp_ok`,
  `render_cells`, `nav_rows`, `listing_cells`) are the standing guard against
  "faster because it did less".

## Two "predict what changed" schemes, both measured and rejected

Both would have made a rename nearly free. Both fail for the same reason: **IDA
and Hex-Rays drift on their own**, so "what the edit changed" is not the same
question as "what is different now".

- **Listing rows, predicted from `xrefs_to` + the function's extent.**
  `/tmp/whatchanges.py` rebuilds the segment before and after and diffs every
  row. echo: 19/19 changed rows covered. ls_ttl: 53/54 — the miss was
  `lea rcx, unk_1D7A0` → `byte_1D7A0`, which the rename did not cause; IDA's own
  analysis defined that byte.
- **Decompilations, predicted from "the old name appears in the cached text".**
  `/tmp/decchanges.py` decompiles 25 functions, renames one, recompiles all and
  diffs. **16 misses over 4 renames, every one of them Hex-Rays' type inference
  moving** — e.g. `unsigned __int64 f(..., unsigned int a4)` → `..., int a4)` in
  functions with no connection to the rename.

The lesson generalises: predicting the effect of an edit on a database that has
its own opinions is unsound. Verify instead — `heads(digest=True)` works because
it asks what the row renders as *now*, not what should have changed.

(This also explains the `lg_decomp_lines` drift blamed on CPU starvation in v5
#4: it is probably the same Hex-Rays instability.)

## Added late in the session

- ~~**Refresh only the rows a rename actually changed** (xrefs-driven).~~
  **MEASURED AND REJECTED.** `/tmp/whatchanges.py` rebuilds the whole segment
  before and after a rename and diffs every row. echo: all 19 changed rows over
  4 renames were covered by (function extent + `xrefs_to`). ls_ttl: 53 of 54 —
  and the one that was not, `lea rcx, unk_1D7A0` → `byte_1D7A0`, **was not
  caused by the rename at all**: IDA's own analysis defined that byte. Any
  address-predicted invalidation leaves such a row stale for good. Superseded by
  the digest scheme, which is exact because it looks at the rendered line.
- **Features the bench still doesn't drive end to end**, in the order they seem
  worth probing: xrefs (`x`), the strings browser (`"`), literal formats (`o`),
  make-code/data/function edits, history (`back`), execution traces, the RPC
  layer. Use the `/tmp/featprobe.py` shape: call the domain API for each and look
  for a number that is absurd for the work done. That is how the flowchart hull,
  `decomp_map` and the rename walk were all found.

## The UI is much slower while a big segment streams (measured, not fixed)

Opening a big binary leaves `ListingView._grow` streaming the segment in the
background for ~7s. During that window every UI action is several times slower:
an xrefs dialog measured **691ms while streaming against 105ms after** on bash.
Two mechanisms, both confirmed:

* **`@work(exclusive=True)` does not stop a thread worker that is already
  running**, and navigating inside the same segment re-primes against the SAME
  model — so every jump leaves another streamer behind, all queueing on the
  model's load lock. Five jumps into bash meant five streamers.
* Each streamer reports growth to the UI every four pages; a report is a thread
  hop, a `virtual_size` change and a full repaint.

Two fixes were tried and **both measured worse**, so neither was kept:
a time-based report throttle (10/s) made it worse because the count is per
*streamer* and there are several; adding a token so only the newest streamer
survives did not reduce the report count either, which means the retirement is
not happening where it looks like it should — worth understanding before trying
again. Probe: `/tmp/streamresp.py` (navigate + press `x` repeatedly while
`lst.model.complete` is still False, counting `_grew` calls).

Caveat: these numbers were taken on a loaded box and the probe re-primes the
view on every iteration, which is itself what spawns the extra streamers. Build
a cleaner probe first.

## Pre-existing crash, unrelated to performance

`StringsPalette.on_mount` calls `self._apply("")`, which does
`query_one(OptionList)` before `compose`'s children are mounted:
`NoMatches: No nodes match 'OptionList' on StringsPalette()`. Reproduces 3/3 on
`targets/bash` with `/tmp/strcrash.py`, **and 3/3 on the pre-autoresearch commit
2b0ae8d** — so it is not something this work introduced. `ProjectPalette` has the
same shape and the same latent race.

## Trace memory reads scale with trace LENGTH (measured, not fixed)

`idatui/trace.py` loads linearly (36.6 / 73.3 / 143.9 / 280.7 ms for 20k / 40k /
80k / 160k rows — x1.95 per doubling, exactly right) and `register_state` is
effectively O(1). But `Trace.memory(addr, length, idx)` costs 16.3 / 31.7 / 63.4
/ 125.8 ms for 200 calls over those same traces: **linear in trace length per
call.**

`_mem_index` sorts accesses by address and bisects to the window, which is the
right idea — but it then iterates *every* access in that address window across
all time, filtering by `t > idx`. A hot stack slot in a loop is written once per
iteration, so the stack pane's cost grows with how long the trace ran. On a
10M-instruction trace a single step could scan millions of entries.

The fix is to find, per byte, the latest access with `t <= idx` rather than
scanning them all. The sort is already stable, so entries within one address are
in time order — but accesses have variable length and overlap, so grouping is
not trivial. Probe: `/tmp/traceprof.py`.

**Do not attempt this until `tests/test_trace_vs_tenet.py` can run** — it is the
differential against Tenet's own reference reader and it is currently skipped
here, which leaves `tests/test_trace.py`'s 35 synthetic checks as the only guard
on a subtle indexing change.

## A bench phase for item edits hangs (attempted, reverted)

`bump_items(ea)` keeping the listing's walk is worth 257x (4890ms → 19ms on
bash) but is **not visible in total_ms**, because the bench has no item-edit
phase. One was written and reverted: driving `undefine` from inside the pilot
hangs the run (no output, killed at the timeout), while the identical sequence
against `Program` directly is fine, and the same sequence with a `print` between
`prog.listing(ea)` and `ensure_ea` is also fine.

**That explanation was wrong** — corrected by a stack dump
(`faulthandler.dump_traceback_later`, `/tmp/hangdiag2.py`). At the moment of the
hang there are **no idatui threads at all**: the main thread is idle in
`selectors.select()` and the only others are idle asyncio executor threads. The
app is stuck *before* `app.run_test()` even returns — nothing to do with
`bump_items`, `_prime`/`_grow`, or `_load_lock`.

It is **pilot start-up flakiness**, and it is partly environmental: several
orphaned `idatui/worker.py` processes had accumulated from runs killed by
`timeout`, and clearing them (`pkill -f idatui/worker.py`) made the next run
boot fine — but it recurred afterwards, so that is not the whole story. Not the
kitty-graphics query either (`IDATUI_KITTY=0` still hangs).

Two things to take from it: **kill stray workers between probe runs**, and a
bench phase should not be built on this until app start-up under the pilot is
reliable. The item-edit win is carried by `/tmp/itemedit.py` (direct
measurement) and `.auto/check_edit.py` (correctness, in the gate).

## decomp_map: what is actually left (measured, corrects run #30)

Run #30 said "what is left in `decomp_map` is `ida_hexrays.decompile`, which
duplicates the decompile the view already did". **That is wrong.** A warm
`ida_hexrays.decompile()` is **0.01 ms** (`/tmp/hxcache.py`) — Hex-Rays' own
cache is free, and the duplicate costs nothing.

The sweep's real split, over 15 417 lines of bash (`/tmp/sweepprof.py`):

| part | cost | calls |
|---|---|---|
| `dstr()` | **2 581 ms (79%)** | 106 594 @ 24.2 µs |
| `get_line_item` | 691 ms | 445 337 @ 1.55 µs |
| `tag_remove` for the length | 14 ms | 15 417 |

Fixed by memoising `obj_id -> ea` for the whole function (v7 #44). What remains
is `get_line_item` per column, which is a real probe per screen column.

Ideas for the remainder, in order of appeal:

- **Map only the lines the split view can show.** The pane paints ~40 lines but
  the map is built for all 3 486. This is the same "compute on demand" shape
  that won for search highlight ranges (v5 #6). It needs a windowed tool
  (`first`/`count`) and a lazy container, because `app.py` and `trace_ctl.py`
  both index the whole list.
- Stepping over columns instead of probing each one is **not** safe: an item
  occupying one or two columns (a single-character variable) would be skipped
  entirely, silently dropping an EA from the region highlight.

## Re-printing instead of re-decompiling after a rename (measured, NOT applied)

`cfunc.refresh_func_ctext()` on a cached ctree is **46x faster** than the
recompile a rename currently forces (31 ms vs 1 432 ms for ten functions), and
it is arguably what a user expects: only the name changes.

**Not applied, because it changes what is on screen.** Only 2 of 10 functions
re-printed to the same text as a real recompile; the other eight differ by
Hex-Rays *type inference*: `char *` vs `const char *`, `__int64` vs
`signed __int64`, `unsigned int a4` vs `int a4`. Same drift already recorded
above — a full recompile re-runs inference with more accumulated knowledge, so
the two disagree even where the rename is irrelevant. Choosing the stabler text
is a product decision about what the pseudocode pane should show, not a
performance change, so it needs a human call. Probe: `/tmp/reprint.py`.

Also checked and **not** a bug: the split view's text and its `decomp_map` do
come from the same ctree. `Program.decompile` calls the `force_recompile` tool
(which does exist) before refetching, so the tool's plain
`ida_hexrays.decompile()` repopulates the cache and `decomp_map` then hits it.
`/tmp/mapalign.py` appeared to show a mismatch only because the probe itself
used `DECOMP_NO_CACHE`, which the app never does.

## PARKED: a 3.6x faster decompile_function_safe (measured, byte-identical, discarded on the metric)

ida-pro-mcp's `decompile_function_safe` — the function that produces the
pseudocode the pane shows — has the **same three faults** that were fixed in
`decomp_map`:

* it allocates **three** `ctree_item_t` SWIG objects per pseudocode line, and
  `_head` and `_tail` are never read (`get_line_item` takes None for both);
* it calls `dstr()` per line to recover the `/*0xEA*/` marker — 24 µs a call —
  where consecutive lines of a multi-line expression report the same ctree item,
  so memoising by `obj_id` (unique within a cfunc) skips most of them.

Measured, on a **warm** cfunc (so this is pure post-processing, no Hex-Rays):

| workload | before | after |
|---|---|---|
| bash's 8 largest, 18 991 lines (`/tmp/decprof.py`) | 2 302 ms (121.2 µs/line) | 429 ms (22.6 µs/line) |
| the same through the real worker (`/tmp/verifybind.py`) | 2 695 ms | 746 ms |
| echo's 12 largest, 3 298 lines (`/tmp/splitprof.py`) | 219 ms | 120 ms (30 µs/line) |

Byte-identical: `check_decomp.py` runs both implementations against the same
cfunc with `include_addresses` both ways — **128/128 echo and 372/372 ls_ttl**.
It is a real gate: keying the memo on `it.op` instead of `it.obj_id` fails 69 of
128.

**Discarded anyway**, because it does not move `total_ms`. Three runs with it
(25 367 / 25 058 / 25 171) against three without (24 514 / 25 122 / 25 783) —
the means are 25 199 vs 25 140, i.e. indistinguishable. The reason is in the
third row of the table: the saving is 98 µs/line on bash's *largest* functions
but only 30 µs/line on small ones, and the bench's fixed set averages 286 lines
a function. Expected effect ~150–250 ms against a run-to-run spread of ±400–600
on this box.

**It is still a real win for the operation a user waits on** — an F5 on a
2 374-line function drops 287 ms → 54 ms of post-processing — so it is parked
rather than deleted:

* `.auto/parked/fast_decompile.patch` (applies to `server/patch_server.py` and
  `idatui/worker.py`)
* `.auto/parked/check_decomp.py` (re-wire into `checks.sh` if the patch is
  re-applied; it will crash if run without it, since it slices the function out
  of `BODY`)

Re-apply it if the benchmark ever decompiles large functions, or if the goal
moves from total session time to per-operation latency. Do **not** re-shape the
bench's fixture set to make this win visible — that would be fitting the
benchmark to the change.

## PARKED: pc_nums, the third instance of the same bug (2.03x, byte-identical)

`_idatui_pc_nums` allocated **three `ctree_item_t` SWIG objects per candidate
column** — inside a scan that probes every literal-looking character of every
pseudocode line. `a`–`f` are hex digits, so `a1`, `v6` and `sub_1F4C0` all
qualify and most columns of a line get probed. `head` and `tail` were never
read. This is the same fault as `decomp_map`'s sweep and
`decompile_function_safe`'s loop — **three instances of one bug**.

It is on the F5 path: `app.py:_load_decomp` fetches `pc_nums` after every
successful decompile so the view can mark the literal under the cursor without a
round trip per keypress.

Measured warm (Hex-Rays already cached), bash's 8 largest, 18 991 lines:
**1 247 ms → 614 ms (2.03x), 66 → 32 µs/line**, with the literal count identical
(7 015). Also stopped `tag_remove` running twice over every line (`pc_nums` and
`_idatui_pc_nums` each called it); worth ~nothing on its own but it is strictly
less work.

Equivalence: the tool's whole output dumped per function and compared across
revisions (`/tmp/pcnumdump.py`, cross-process — an in-process differential
**segfaults**, two SWIG item objects over one cfunc). echo: 128 functions,
1 463 literals, **0 mismatches**. `tests/test_scenarios.py` also drives literal
cycling through these exact column extents.

Parked at `.auto/parked/fast_pc_nums.patch` for the same reason as the decompile
patch: real work removed, but ~170 ms against a ±500 ms run-to-run spread.

## Both parked patches, measured together

Applied together (they are both on the F5 path) over four runs: 25 367 / 25 058 /
25 171 / 24 585, against three without: 24 514 / 25 122 / 25 783. Means **25 045
with, 25 140 without** — a 95 ms edge inside a 500 ms spread, i.e. still not
resolvable. Discarded on the metric, kept on disk.

Apply both if the goal moves to per-operation latency: an F5 on a 2 374-line
function loses ~233 ms of text post-processing and halves its `pc_nums` cost.

## A 27 283 ms outlier, and how to recognise one

One run came back at 27 283 (against ~24 500) with `lg_split` 2 200 → 6 917 and
`lg_search` 3 441 → 840. It was NOT the change under test: the **work counters
moved with it** — `lg_decomp_lines` 3 436 → 3 233, `lg_split_mapped_lines`
2 070 → 1 993, `lg_search_hits` 91 783 → 92 733. Hex-Rays decompiled bash
differently that run (the drift documented above), which changed how much of the
post-rename re-render the split phase absorbed before search got to it. The next
run reproduced 24 585 with every counter back to its usual value.

**The NOTES counters are what tell an outlier from a regression.** A real
regression moves the time and leaves the work alone.