path: root/.auto/prompt.md

# Autoresearch: make ida-tui faster, without losing anything

## Objective

Reduce the wall-clock latency of the operations an ida-tui user actually waits
on, on both a small binary (`targets/echo`, 128 funcs) and a real-world one
(`targets/bash`, 2099 funcs). No feature may be removed, no output may change,
no test may break.

The app is a Textual TUI over a private idalib worker process (unix socket,
length-prefixed pickle). Three layers, kept separate:

- `idatui/worker.py` + `idatui/worker_client.py` — backend; one call = one
  round trip to a process that owns the IDA database.
- `idatui/domain.py` — paging/caching over the client (`FunctionIndex`,
  `ListingModel`, `DisasmModel`, `HexModel`, `decompile`, xrefs, resolve).
- `idatui/app.py` — the Textual app; views are line-virtualized `ScrollView`s.
- `idatui/graph.py` — pure-python Sugiyama layout for the CFG view.
- `server/patch_server.py` — the extra `@tool`s the worker injects into
  ida-pro-mcp (`heads`, `read_raw`, `resolve_names`, …). This is where a new
  backend capability goes; it runs INSIDE the worker with full idalib access.

## Metrics

- **Primary**: `total_ms` (ms, lower is better) — the sum of every phase median.
  Re-baselined once (experiment v2 #1) when `phase_graph` was found to be timing
  a cache hit; the v1 history below is still the record of what was learned.
- **Secondary** (all in ms, per target: `sm_` = echo, `lg_` = bash):
  - `nav_ms` — jump to a function's entry in the listing, cold. **The single
    biggest term today** (`lg_nav_ms` ≈ 28 s of a 46 s total, and
    `lg_nav_worst_ms` ≈ 28 s for ONE jump).
  - `search_ms` — incremental search over a whole segment.
  - `decomp_ms` — F5 → decompile → highlight → paint (nav excluded).
  - `graph_ms` — flowchart → layout → paint.
  - `boot_ms`, `listing_cold_ms`, `listing_warm_ms`, `render_ms`, `hex_ms`,
    `index_ms`, `palette_ms`, `pure_graph_ms`.
  - `fails` — **must stay 0.** A phase that silently stops doing its work would
    otherwise read as an enormous speedup.
- `NOTES` on each run carries the work actually done (`decomp_ok`, `graph_ok`,
  `graph_blocks`, `search_hits`, `listing_rows`, `nav_rows`, `render_cells`, …).
  **If a metric drops and its NOTES counter drops with it, that is not a win.**

## How to Run

`./.auto/measure.sh` (~45 s). Prints `METRIC name=value` lines.
`./.auto/measure.sh --only sm` benches just the small target while iterating.

`./.auto/checks.sh` runs automatically after every passing benchmark (~160 s):

1. `.auto/check_search.py` — the search fast paths against the plain per-line
   loop, for every typed prefix.
2. `.auto/check_rename.py` — the listing after a rename, read narrow (painting)
   and wide (search body), and the whole model against a rebuild.
3. `tests/run.py` — every suite, 830 checks.

**The two `.auto/check_*.py` scripts exist because the things they guard fail
silently.** A stale cache still returns *an* answer, and the benchmark rewards
it for being fast. Twice this session the honest change measured worse than the
broken one. If you optimise a cache, write the check that fails on the old code
first.

## Files in Scope

Anything under `idatui/` and `server/patch_server.py`. In rough order of
expected payoff:

- `idatui/domain.py` — `ListingModel` is where nav time lives. It walks the
  segment forward in 500-head pages from `seg_start`, so `ensure_ea(ea)` is
  O(distance from the start of the segment): landing on a function near the end
  of bash costs ~440 sequential worker round trips. `DisasmModel`,
  `HexModel`, `FunctionIndex` are the other paging/caching classes.
- `server/patch_server.py` — the `heads` tool the listing pages over. A better
  backend primitive (address-anchored start, bigger/denser pages, a count-only
  or index mode) is fair game and probably the real fix.
- `idatui/app.py` — `render_line` of `ListingView`/`DecompView`/`HexView`/
  `GraphView`, the search mixin (`_compute_matches`, `_line_plain`), `_grow`,
  `_prime`.
- `idatui/graph.py` — layout; already Fenwick-optimised once (see the idatui
  skill), so the easy win is gone.
- `idatui/worker.py`, `idatui/worker_client.py` — transport (pickle framing,
  per-call overhead).
- `idatui/highlight.py` — Pygments C lexing per decompilation.

## Off Limits

- `tests/**` — the correctness gate. Do not weaken, skip, shorten or "fix" a
  test to make a change pass. If a test fails, **assume the change is wrong**.

  Two narrow exceptions have been used, each with proof recorded in the log:
  the racy-setup one below, and adding `.auto/check_*.py` gates (which only ever
  *add* coverage).

  The racy-setup exception, and what it costs you: a scenario whose *setup* is
  racy, where the speedup merely decides which of two async loads lands first.
  Before touching it you must (a) bisect to show which change flips it, (b)
  reproduce the race outside the suite, showing the app reaching two different
  states from the same steps, and (c) show the repaired scenario passing on
  BOTH the fast and the slow code. Only the setup may change — every `c.check`
  stays exactly as it was — and the ASI must record all three proofs. Done once
  so far, for `graph_minimap` (experiment #6).
- `.auto/bench.py` may only be changed to add *more* signal (extra metrics,
  extra NOTES). Never to do less work, shorten a sweep, drop a phase, loosen a
  wait, or pick easier functions. If you change what it measures, say so in the
  log and re-baseline with `init_experiment`.
- `targets/**` binaries and their `.i64`/`.pristine.i64` databases.

## Constraints

- **No functionality may be lost.** Same rendering, same colours, same
  behaviour. `checks.sh` must pass.
- **No new third-party dependencies.** `domain.py` and `worker_client.py` are
  deliberately stdlib-only (the TUI layer may use Textual/Rich/Pygments, which
  are already dependencies).
- **No caching that can go stale silently.** Renames bump `Program._name_gen`
  and disasm caches are cleared for a reason; a new cache must have an
  invalidation story or it will show stale names after an edit.
- The worker is single-threaded and main-thread-only for idalib. Parallelism
  has to come from batching calls, not from calling IDA concurrently.
- Do not tune constants to the two benchmark binaries. A change must be a
  structural improvement that holds for a 10 MB firmware image too.

## What's Been Tried

v1 bench baseline `total_ms` ≈ 46 600 → 18 900 after thirteen experiments
(−59% on v1, then a re-baseline at 27 913 and −32% on v2).

**Wins, biggest first**

1. *(v1 #3, −42%)* **ida-pro-mcp installs a `sys.setprofile` hook around every
   tool call.** Its deadline mechanism profiles every python call/return so a
   pure-python tool body can be interrupted — a 3.3× tax on a backend whose
   tools are call-heavy (`heads`: 92 → 28 µs/row without it). `worker.py` now
   sets `IDA_MCP_TOOL_TIMEOUT_SEC=0` and arms the deadline itself with one
   polling watchdog thread + `ida_kernwin.set_cancelled()` — the half that
   actually frees the IDA main thread.
2. *(v6 #2, −41%)* **`decomp_map` swept every column three times over.** Three
   SWIG allocations per column, the TAGGED line length as the bound (124 columns
   for a 23-column line), and a `dstr()` format per column when consecutive
   columns are the same ctree item. bash's 25 largest: 67.4 s → 6.8 s. This is
   the split view's whole cost.
3. *(v7 #2, −23%)* **A rename kept the listing's walk.** `bump_names` discarded
   the segment model, so the reload re-walked it to find a row the cursor was
   already on — 1.7 s per rename on bash. Now the walk stays and the text is
   re-rendered a block at a time: 10.3 ms.
4. *(v2 #2, −32%)* **`Program.flowchart` fetched the convex hull of the basic
   blocks.** IDA function chunks live far from the entry, so a 1.4 KB function
   could span 680 KB: 128 000 rows fetched, 3 s to draw, and the far blocks
   came back *empty* because the pager's 64-page bound ran out first. Now it
   fetches the merged block intervals and assigns rows by bisect.
5. *(v1 #6)* **`lru_cache` on the per-line render** (`_idatui_line_parts`):
   196 k listing lines of bash are only 53 k distinct, 26.7 → 16.5 µs/row.
6. *(v1 #7, −9%)* **Incremental search narrows instead of rescanning** — typing
   a character can only remove lines.
7. *(v5 #6)* **Highlight ranges are computed per line on demand.** Searching
   one character over bash matches 177 k lines at 310 k places; all but the
   forty on screen were built and thrown away.
8. *(v7 #10, −4%)* **`heads(digest=True)`** — ask whether a page still renders
   as you hold it, rather than fetching it to find out.
9. *(v7 #12, −2.6%)* Three redundancies in the `heads` walk: item flags fetched
   three times per head, `get_func` per head where a head is nearly always in
   the same function as the one before it, and the page digest rebuilding a
   tuple-of-tuples per row where one spans list is shared by ~45% of them.
10. *(v1 #9/#10/#13, v2 #3, v5 #3)* Constants: `bytes.hex(" ")` for the opcode
   column (12×), `bisect` imported at module scope, the deferred
   `refresh(layout=True)` only when a scroll actually clamped, a memoised
   pygments token→style lookup, `_CellRow` writing by slice, `HexView` emitting
   style runs instead of a Segment per byte, `Head` as a `NamedTuple`, and the
   graph's transposition counting keep and swap in one pass.

**Dead ends / things not to re-try**

- `re.finditer` per tag in the span walker is *slower* than a plain character
  loop (14.4 vs 13.2 µs/line): Match objects cost more than the ~54 trivial
  iterations they replace. One capturing `re.split` is what wins (10.2).
- `ListingModel.PAGE` (500 / 1000 / 2000) makes no measurable difference —
  the cost is per row, not per round trip. Don't tune it.
- Changing the `heads` wire shape (int `ea`, tuples instead of dicts) buys at
  most 0.3 µs/row. Measured; not worth a breaking change.
- A page-scoped identity memo for the spans→tuple conversion never hits: the
  repeats are spread across the segment, not within a 500-row page. A
  model-scoped one is unsafe (`id()` is reused once the page dicts die).
- Growing the line cache past 16 384 does nothing for a *cold* sweep (17.0
  µs/row at 16 k, 32 k, 64 k and 128 k alike). It only helps a *second* sweep —
  which was a dead end until the rename fix created one, and then it was worth
  21%. **Re-read the dead ends after a structural change: this one stopped being
  one.** Sized at 65 536 now (bash's .text has 53 363 distinct lines; 32 768
  still thrashes). Costs +47 MB of worker RSS, and it is a bound rather than a
  proportion — a bigger binary fills it and stops.
- Merging same-kind adjacent spans in `_idatui_spans`: only 2.5% fewer spans on
  20 k real lines. Not worth a wire-format change.
- Applying an already-decompiled function inline instead of via a `@work`
  thread: a Textual thread spawn plus its `call_from_thread` is worth well under
  1% of an F5. Measured flat; reverted for complexity.
- Deferring the "decompiling…" loading cover until ~120 ms (worth ~9 ms per F5)
  is blocked: a scenario asserts F5 raises it **synchronously**, guarding a real
  past regression. That is an assertion, not setup, so it stands.
- Polling the worker socket faster than ~25 ms is actively harmful: the poll
  runs on a background thread and starves the UI thread through a cold
  auto-analysis (see v5 #4).

**Where the time is now (28 842 ms on the v7 bench)**

`lg_search` 7 123 · `lg_nav` 6 610 · decomp lg+sm 3 647 · split lg+sm 4 027 ·
graph lg+sm 1 691 · boot lg+sm 1 151 · listing lg+sm 1 428 · rename lg+sm 1 119 ·
hex lg+sm 895 · render 483 · pure_graph 212.

`lg_search` is large *because of where it sits in the session*: the bench renames
six functions and then searches the whole 228 k-row segment, so it pays to
re-render everything the rename staled. Before the rename fix that same cost was
paid up front, inside the renames (`lg_rename` was 10 055 ms). Re-rendering N
heads costs what loading N heads costs; the win was in not doing it for rows
nobody reads.

**Five things are at a floor that is not ours to move:**

- `nav` — `generate_disasm_line` is 5.9 µs of the ~16.5 µs/row the worker
  spends, and the walk is inherently linear.
- `decomp` — the raw `decompile` tool is 1 711 ms cold for echo's twelve largest
  functions and 218 ms warm; pickling the result is 0.1 ms. It is Hex-Rays.
- `search` — after a rename, what you read has to be re-rendered. The worker
  still has to *render* a page to know it is unchanged (`generate_disasm_line`
  is the floor), but since v7 #10 it no longer has to ship it: `heads(...,
  digest=True)` returns hash+count, and the client keeps the page it already
  has. That is ~40% of a page's cost, and after a rename nearly every page is
  unchanged.
- `listing`/`hex`/`graph`/`render` — mostly Textual's own compositing, ~6 ms per
  full-screen frame. Our `render_line` is ~1.8 ms of a ~10 ms hex frame.
- `boot` — ~150 ms of it is the worker importing `idapro`.

**Benchmark history.** The bench was corrected five times. Three of those found
a cost that was *entirely invisible*, and two of those three turned out to be
among the largest wins of the whole session. Every re-baseline is in
`.auto/log.jsonl`:

| bench | baseline | best | what changed |
|---|---|---|---|
| v1 | 46 572 | 19 006 | — |
| v2 | 27 913 | 17 501 | graph opens were timing a **cache hit** |
| v4 | 18 498 | — | decomp/search/index reps were timing cache hits |
| v5 | 18 516 | 17 465 | landing polls every 2 ms, not 10 |
| v6 | 33 502 | 19 835 | **split view was not covered at all** (48% of a session) |
| v7 | 33 243 | 28 651 | **rename was not covered at all** (1.7 s each on bash) |

Headline user-facing numbers, measured directly rather than through the bench:
a cold jump to a far address on bash 28.3 s → 6.4 s; drawing a chunked
function's graph 3.0 s → 0.01 s (and its far blocks are no longer empty);
opening the split view 850 ms → 110 ms; getting the listing back after a rename
5.7 s → 10 ms.

**The most productive thing in this session was asking what the bench does not
measure.** Features still uncovered: xrefs (`x`), the strings browser (`"`), the
struct editor, literal formats (`o`), make-code/data edits, history, execution
traces, the RPC layer. Domain-level probes say xrefs/strings/structs/resolve are
all fast (`/tmp/featprobe.py` pattern), but nothing has driven them end to end.

**Measurement traps**

- **Check `uptime` before believing a number.** This box started the session at
  load 0.6 and drifted to 1.6; `lg_nav` (224 000 sequential worker round trips)
  moved 6 556 → 7 058 with no code change, while the direct paging microbenchmark
  stayed at 24.4 µs/row. Totals are not comparable across a load change — rebase
  on a fresh same-session run, or verify in a microbenchmark:
  `/tmp/domain_break.py` (cold paging), `.auto/diff_spans.py` (span walker),
  `tests/test_graph.py <corpus>` (layout).

- `cProfile` massively distorts this code (it is call-heavy): it reported
  `_idatui_spans` at 68% of the `heads` tool when the real share was ~10%.
  A/B with `time.perf_counter` in one process instead.
- Anything that walks the listing twice in one process is measuring a warm
  `_idatui_line_parts` cache the second time. Run cold cases first, or in
  separate processes.
- Anything that goes through `Program.flowchart`, `Program.decompile` or
  `Program.functions` twice is measuring a cache. Clear it or use the raw tool.
- **The worker is a single serial process.** Work "moved to the background"
  does not overlap with anything; only doing less work helps.