ida-tui.git/idatui/worker_client.py, branch main

projects: worker pool with memory-budgeted residency (phase 1b)

2026-07-25T09:29:36Z

idatui/pool.py — WorkerPool keeps one live worker per project binary: * lazy spawn on first use; staging + a scratch sweep happen first, so a database wedged by a previously hard-killed worker reopens instead of crash-looping. * residency is bounded by a MEMORY BUDGET (default project.memory_pct of RAM), not a worker count — a count is the wrong knob when one project holds a 50KB helper and a 6MB crypto lib. Cost is measured per worker from /proc//smaps_rollup (PSS, which splits shared pages so summing means something). * over budget -> evict least-recently-used, never the active or pinned binary, and give up rather than thrash when nothing is evictable. Eviction calls idb_save first, so returning to a binary is a DB load, not a re-analysis. * status() feeds the switcher UI (resident/pinned/active/analysed/memory). worker_client: fix the wedge-file bug this depends on. close() sent __shutdown__ and then IMMEDIATELY SIGTERMed, killing the worker mid close_database() — which is what leaves the unpacked .id0/.id1/... behind and makes the .i64 refuse to reopen. Now it waits out a grace period (the worker returns from serve() on __shutdown__ and closes the DB in its finally) and only escalates if it's genuinely stuck. Also expose .pid for memory accounting. Verified: a pilot run that used to leave echo.id0/.id1/.id2/.nam/.til now leaves only echo.i64, and teardown is no slower (14 checks in 5s). tests/test_pool.py: 22 checks with an injected fake client — LRU order, budget eviction, active/pinned protection, save-before-close, thrash avoidance, status(), teardown. Pure stdlib, no idalib.

refactor: extract shared error hierarchy + Session into idatui/errors.py

2026-07-24T12:41:23Z

The IDAError/IDAConnectionError/IDAToolError/... exceptions and the Session dataclass were defined in client.py (the ida-pro-mcp HTTP client), but the idalib worker path (worker_client/domain/app) needs them without the HTTP transport. Move them to a transport-agnostic errors.py; client.py re-exports them so the deprecated mcp tooling and stress tests are unchanged (verified: errors.IDAToolError IS client.IDAToolError, so cross-module `except` still works). worker_client, domain (TYPE_CHECKING-guarded IDAClient hint), app, and __init__ now import the shared types from errors.py. This decouples the worker path from client.py at runtime -- the prerequisite for deleting the mcp transport.

worker: spawn under the IDA python (has ida_pro_mcp), not the TUI's

2026-07-23T23:47:05Z

Root cause of "ModuleNotFoundError: No module named 'ida_pro_mcp'": the worker was spawned with sys.executable — the TUI's python (~/ida-venv) which has idalib + textual but NOT ida_pro_mcp. The package split on this box: /usr/bin/python : idapro + ida_pro_mcp (the "IDA python") ~/ida-venv/python : idapro + textual (the "TUI python", runs the app) Fix: WorkerClient now auto-detects a python that can import ida_pro_mcp (IDATUI_WORKER_PYTHON override, else /usr/bin/python[3], else sys.executable) and runs worker.py as a SCRIPT rather than `-m idatui.worker`, so it doesn't import the textual-dependent idatui package __init__ under a python that has no textual. worker.py itself is pure stdlib at load; idapro/ida_pro_mcp are imported at runtime (both present in the IDA python). Verified: detection returns /usr/bin/python; worker.py loads clean there.

worker: surface the real startup failure (not just "code 1")

2026-07-23T23:42:43Z

The worker's stderr was swallowed by the TUI, so an open failure showed only "worker exited during startup (code 1)". Now: * WorkerClient captures the worker's stdout+stderr to /tmp/idatui-worker-*.log and, on a startup exit, surfaces the last meaningful line in the error (the worker prints a clean 'WORKER-FATAL: ...' marker; _log_tail prefers it). * worker.py wraps main() to print that marker + traceback before exiting 1, and gives an ACTIONABLE open error: "failed to open : the .i64 is likely held by a running ida-mcp worker (pkill -f idalib) or wedged (delete .id0/.id1/ .id2/.nam/.til)". Also calls ida_auto.auto_wait() after open to fully match ida-mcp's session manager (open_database + auto_wait). Root cause of the reported failure is almost certainly a leftover ida-mcp worker still holding bash's .i64 from earlier --backend mcp runs: idalib can't open a database another process has locked. Fix: pkill -f idalib, then retry --backend worker; the error message now says so instead of "code 1".

worker: idalib worker + WorkerClient (drop-in for IDAClient) — migration step 1

2026-07-23T23:29:06Z

First concrete step off the mcp HTTP transport. Instead of reimplementing ~25 tools, reuse ida-pro-mcp's tool *functions* verbatim and replace only the transport + process management: * idatui/worker.py — opens ONE database in-process on the main thread (as idalib requires), imports ida_pro_mcp (which registers every stock + our patched-in custom tool against MCP_SERVER), then serves MCP_SERVER.tools.methods[name] (**args) over a unix socket with length-prefixed pickle. Serial on the main thread (idalib is single-threaded; tools run inline through execute_sync). Session-management tools (idb_open/idb_save/server_health/idb_list) are shimmed since the worker *is* the single session. * idatui/worker_client.py — WorkerClient exposes the exact surface the app/domain use on the client (call/call_envelope/connect/set_db/resolve_db/list_sessions/ health/keepalive/close) and returns byte-identical payloads (the worker calls the same functions IDAClient.call ultimately hits). So domain.py and the app are UNCHANGED — you just construct a WorkerClient instead of an IDAClient. Calls are serialized under a lock over one socket; keepalive is a no-op (the worker is ours and never idles out). Not wired into the app yet — the mcp path is fully intact. Verified without idalib: pickle framing round-trips arbitrary payloads incl raw bytes; WorkerClient has full IDAClient surface; call_envelope produces the result.structuredContent shape domain.decompile() reads. The idalib E2E (experiments/worker_smoke.py drives the real domain.Program read path through the worker) is written but couldn't run here — this sandbox has degraded to reaping any idalib spawn; the underlying unix-socket protocol already ran clean in the inproc_spike bench (~50us/call), and the worker dispatches the same tool functions the HTTP path does, so shapes match by construction. Next: stand up progress reporting during analysis, then flip _connect/_reconnect to build a WorkerClient behind a flag and run the pilot suite against it.