1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
|
# Paging & scale findings (the "millions of lines" problem)
Measured against a real target: `libcrypto.so.3` (5.7 MB, **10,092 functions**,
biggest function **52,120 instructions**). These constraints drive the domain /
paging layer. They describe the ida-pro-mcp *tool functions* (`list_funcs`,
`disasm`, `decompile`, `xref_query`, …) which the idalib worker now calls
in-process (`idatui/worker.py`) — the shapes and caps below are the tools'
behaviour and are unchanged by dropping the HTTP transport.
## Response shape (list_* / *_query tools)
A query tool payload is a *list of per-query results*:
```json
{ "result": [ { "data": [ ... ], "next_offset": <int|null> } ] }
```
## Per-call caps are silent and dangerous
Every `count` / `max_instructions` parameter is honored up to a cap, then
**silently collapses to a tiny default (10)** — it does NOT clamp to the max.
| tool | param | honored up to | above cap |
|-------------|------------------|---------------|-----------|
| `list_funcs`| `count` | ~700 | → 10 |
| `disasm` | `max_instructions`| ~500 | → 10 |
**Rule:** clamp page sizes client-side. Use `list` page ≤ **500**, disasm window
≤ **500**. Never pass a large count hoping for clamping.
## Pagination: advance by `len(data)`, NOT `next_offset`
`next_offset` is just `offset + count`. If `count` exceeds the cap you get 10
rows but `next_offset` still jumps by `count`, **skipping ~99% of the data**.
Correct enumeration:
```python
offset = 0
while True:
data = call("list_funcs", queries=[{"offset": offset, "count": 500}])[...]
yield from data
if len(data) < 500: # short page => end
break
offset += len(data) # advance by what we actually got
```
Full 10,092-function enumeration = 21 pages, **~3.1 s** (0.31 ms/func). Do this
once in the background with a progress indicator; page lazily otherwise.
## disasm offset paging is **O(offset)** — the main scroll hazard
`disasm addr=F offset=N` re-walks from the function start, so latency grows with
depth (window = 60 instrs):
| offset | latency |
|--------|---------|
| 0 | ~6 ms |
| 1,000 | ~10 ms |
| 5,000 | ~23 ms |
| 20,000 | ~75 ms |
| 50,000 | ~180 ms |
There is **no resumable cursor**: the payload's `cursor: {"next": N}` is just
informational; `disasm` has no cursor input param. So O(offset) is unavoidable
via this API.
Implications for the domain/paging layer:
- **Cache fetched windows** by `(func, offset)`. Re-scrolling a deep window
currently re-pays the full ~180 ms (no server-side cache); our cache makes
revisits instant.
- **Prefetch ahead** in background threads (client is concurrency-safe) so the
next window is warm before the user reaches it.
- **Over-fetch** up to the ~500 cap per call and slice locally for the viewport,
amortizing both RTT and the O(offset) walk during fast scrolling.
- **This only bites pathological functions.** The two 52k-insn outliers are
almost certainly mis-analyzed data/unrolled tables; the next largest real
function is 2,748 instrs (deepest offset ≈ 10 ms). Typical scrolling is fine.
## `include_total` is expensive — fetch once, cache
Computing `total_instructions` scans the whole function: **~217 ms** on the 52k
monster (vs ~10 ms without). Fetch total once when a function view opens (to size
the scrollbar) and cache it; never request it per-scroll.
disasm totals are **top-level** fields, not under `asm`:
`total_instructions`, `instruction_count`, `cursor`.
## Decompile: can hard-fail on huge functions (soft error)
`decompile` on the 52k-insn function returns, in ~3 ms:
`{"code": null, "error": "Decompilation failed at 0x3349c0"}` with
`isError == false`. The client surfaces this as **data, not an exception**
(correct). The pseudocode view must handle "decompilation failed" gracefully —
fall back to the disassembly view or show an error panel.
Normal decompile bodies are server-truncated with a `[N chars total]` marker
(still to be solved for full-body display — see Phase 2).
## Worker lifecycle (idatui's own idalib worker)
idatui no longer uses ida-pro-mcp's shared HTTP supervisor. `idatui/worker.py`
opens exactly **one** database with `idapro.open_database(...)` in its own process
and serves tool calls over a unix socket (`WorkerClient`). Consequences vs the
old supervisor model, which several design choices here were built around:
* **No `max_workers` cap, no cross-session contention.** Each TUI owns its
worker; there is no "Maximum idalib worker count reached" and no shared license
slot to free.
* **No idle self-exit / keepalive dance.** The old per-worker `WorkerLifecycle`
watchdog (`idle_ttl_sec`, default 600s) and the `KeepAlive` heartbeat that
fought it are gone with the supervisor. The worker lives as long as the TUI
holds the socket and dies with it. `WorkerClient.keepalive()` is a no-op kept
for API parity, and `--ttl` is passed through but the single owned worker does
not self-reap.
* **A crashed worker drops the socket**, surfacing as `IDAConnectionError`; the
app's `_reconnect` respawns a fresh worker (re-opening + re-analyzing the
binary). The hard-kill lock recovery below still applies.
## Writable path requirement (operational)
`idb_open` writes the `.i64` next to the input binary, so the path must be
**writable**. Opening from read-only dirs (e.g. `/usr/lib`) fails with
`"Failed to open database"`. Copy targets into a writable dir first
(`targets/` in this repo).
|
