aboutsummaryrefslogtreecommitdiffstats
path: root/docs/PAGING_FINDINGS.md (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Rebase MISTER EXO's ida-codemode port onto the current treeblasty39 hours1-27/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Mechanical part of the port: the 27-file patch was cut against a base ~148 commits behind us, so it did not apply. Resolved 11 conflicts (all of them diff drift, not semantic clashes) and the three file deletions: - app.py: the patch re-inserted _do_rename/_do_name_addr/_seek_split etc. as "theirs" because our tree moved them to edit_ctl.py/trace_ctl.py. Kept ours and applied the real intent (WorkerClient->CodeModeClient, .call->.invoke, _open_worker_client->_open_database_client) at their current homes. - domain.py: kept Head as a NamedTuple -- the patch reverted it to a frozen dataclass, which the perf work measured at 2.9us vs 1.9us per row on a quarter-million-row walk. Dropped _fetch_output (no download_url under Code Mode) and its now-dead urllib/json imports. - pane.py: the patch's deletion swallowed our zellij support along with the worker-reaping block it meant to remove. Kept zellij, removed the reaping. - test_scenarios.py: the idb_save->save_database teardown hunk belongs to tests/_fixtures.py in our tree; applied it there and kept our pc_num_format scenario that the drift landed on. Three defects in the patch itself, fixed here: - It made "import idatui" hard-require ida_codemode, so every offline suite died at import -- including the pure ones (graph/index/trace) that are the house rule for "tests/run.py --fast". The import is now deferred and gated on the binding, which is also what lets the port's own contract tests inject a fake DatabaseHandle. - project.stage() inlined an ida_codemode.registry import and treated "library not installed" as "someone owns this database", which broke IDA-free project staging. Ownership lookup moved to codemode_client.database_owner(). - tests/test_codemode_client.py had no NEEDS_IDA marker, which tests/run.py rejects outright. Offline suite: 301 passed, 0 failed. Against master's 344 the whole delta is accounted for: -40 worker_client (module deleted), -18 launch sweep checks (behaviour deliberately removed) +3 guarding that it stays removed, +2 pool (GUI-save semantics), +13 new codemode_client contract tests. NOT yet done, and the port is not functional without it: the adapter is missing five operations our tree grew since the patch's base (flowchart, op_format, pc_nums, pc_num_format, survey_binary) and its "heads" predates back-walking and digest/expect.
* docs: sweep for the worker-only reality (drop mcp supervisor/spawn.sh/--db)blasty2026-07-241-45/+21
| | | | | | | | | | | | | | | | * RPC.md: the TUI is launched via `./ida-tui <bin> --rpc <sock>` (was `idatui.tui --db`); point the "regression lock" note at rpcclient/drive + the pilot instead of the deleted rpc_smoke.py. * PAGING_FINDINGS.md: reframe the intro as the ida-pro-mcp tool functions the worker now calls in-process (shapes/caps unchanged); replace the supervisor-era "idle self-exit" + "max_workers cap" sections with the single-owned-worker lifecycle (no cap, no idle-exit, crash -> reconnect). * TEXTUAL_NOTES.md / TUI_DRIVING_BLUEPRINT.md: drop the ida-pro-mcp framing and the deleted rpc_smoke.py references (-> test_scenarios.py / rpcclient). Also updated the idatui + idatui-rpc skills (in ~/.pi, outside the repo) to the worker model: no supervisor/spawn.sh/--db, worker python + $IDATUI_WORKER_PYTHON, worker.py/worker_client.py/errors.py architecture, and the load-starvation gotcha.
* stop piling up idalib workers; raise max-workers headroomblasty2026-07-091-3/+19
| | | | | | | | | | | | | | | Root cause of 'Maximum idalib worker count reached': the app bumped idle_ttl to ~never on every open, so sessions never freed and hit the default cap of 4. - app: drop the immortal bump_idle_ttl(1e9); rely on the KeepAlive heartbeat (120s) to keep the running session warm, and open with a moderate idle_ttl (1800s) so the worker self-exits and frees its slot after the TUI closes. - spawn.sh: set IDA_MCP_MAX_WORKERS (default 8) for headroom. - doc: supervisor prunes unreachable workers before counting, so killing an idle worker frees a slot on the next open. (Verified separately: big/truncated decompile results propagate renames fine via the earlier force_recompile fix.)
* keep idle workers alive: bump_idle_ttl + KeepAlive heartbeatblasty2026-07-091-8/+29
| | | | | | | | | | | | | | | | Root cause: idalib workers run an idle watchdog (worker_lifecycle.py) that self-exits after idle_ttl_sec (default 600s) of no requests -- deliberate resource hygiene for a shared/ephemeral supervisor, wrong for an interactive TUI. CLI-opened startup binary always gets 600s (no flag). Fix (both verified against a 12s-TTL session): - IDAClient.bump_idle_ttl(1e9): re-open (idempotent) re-applies TTL, no upper cap -> effectively immortal. Primary fix, call once at startup. - IDAClient.keepalive()/KeepAlive: background server_health heartbeat resets the watchdog; covers adopted sessions too. Safety net. tests/test_keepalive.py: 4/4 (heartbeat + bump both keep a short-TTL worker alive past 2x TTL). docs updated with why + fix.
* domain/paging layer: FunctionIndex, block-cached DisasmModel, decompile, resolveblasty2026-07-091-0/+10
| | | | | | | | | | | | | - Program: model registry + small prefetch pool; cache invalidation hooks - FunctionIndex: lazy pagination (advance by len, clamp page=500), filter globs, viewport windows; full 10,092-func enumerate matches survey in ~3.1s - DisasmModel: block-cached windowed disasm (256/block), forward+back prefetch, cached instruction totals; deep window revisit 196ms -> 0ms (cache proven) - Decompilation: truncation-aware, hard-failure surfaced as data (monster funcs) - resolve(): int/hex/symbol via lookup_funcs (string-array query shape) - tests/test_domain.py: 17/17 on both ls (~290 funcs) and libcrypto (10k+52k-insn) - smoke_client no longer assumes 'main' exists (works on libraries) - doc: idle_ttl worker self-exit finding ("not reachable" needs re-open)
* stress paging on 10k-func binary; document scale constraintsblasty2026-07-091-0/+101
Measured against libcrypto.so.3 (10,092 funcs, biggest 52,120 insns): - list_* count cap ~700, disasm max_instructions cap ~500, then SILENT collapse to 10 (not clamped) -> must clamp client-side (use 500). - pagination must advance by len(data); next_offset=offset+count skips data. - disasm offset paging is O(offset): 6ms@0 -> 180ms@50k, no resumable cursor -> domain layer must cache windows + prefetch + over-fetch. - include_total scans whole func (~217ms on monster) -> fetch once, cache. - decompile hard-fails on huge funcs as a soft error (code=null) -> handle. - idb_open needs a writable path for the .i64. tests/stress_paging.py: durable paging benchmark harness docs/PAGING_FINDINGS.md: constraints that drive the paging layer