ida-tui.git/experiments, branch main

worker: idalib worker + WorkerClient (drop-in for IDAClient) — migration step 1

2026-07-23T23:29:06Z

First concrete step off the mcp HTTP transport. Instead of reimplementing ~25 tools, reuse ida-pro-mcp's tool *functions* verbatim and replace only the transport + process management: * idatui/worker.py — opens ONE database in-process on the main thread (as idalib requires), imports ida_pro_mcp (which registers every stock + our patched-in custom tool against MCP_SERVER), then serves MCP_SERVER.tools.methods[name] (**args) over a unix socket with length-prefixed pickle. Serial on the main thread (idalib is single-threaded; tools run inline through execute_sync). Session-management tools (idb_open/idb_save/server_health/idb_list) are shimmed since the worker *is* the single session. * idatui/worker_client.py — WorkerClient exposes the exact surface the app/domain use on the client (call/call_envelope/connect/set_db/resolve_db/list_sessions/ health/keepalive/close) and returns byte-identical payloads (the worker calls the same functions IDAClient.call ultimately hits). So domain.py and the app are UNCHANGED — you just construct a WorkerClient instead of an IDAClient. Calls are serialized under a lock over one socket; keepalive is a no-op (the worker is ours and never idles out). Not wired into the app yet — the mcp path is fully intact. Verified without idalib: pickle framing round-trips arbitrary payloads incl raw bytes; WorkerClient has full IDAClient surface; call_envelope produces the result.structuredContent shape domain.decompile() reads. The idalib E2E (experiments/worker_smoke.py drives the real domain.Program read path through the worker) is written but couldn't run here — this sandbox has degraded to reaping any idalib spawn; the underlying unix-socket protocol already ran clean in the inproc_spike bench (~50us/call), and the worker dispatches the same tool functions the HTTP path does, so shapes match by construction. Next: stand up progress reporting during analysis, then flip _connect/_reconnect to build a WorkerClient behind a flag and run the pilot suite against it.

experiments: add a unix-socket idalib worker as the 3rd bench column

2026-07-23T23:19:31Z

Adds UnixWorkerBackend (Option C): the same DirectBackend, but in a child process that opens idalib on ITS main thread and serves one client serially over an AF_UNIX socket with length-prefixed pickle (bytes ride raw — no hex, no JSON). bench() is now generic over {direct, unix, mcp}; --worker runs the child. 3-way result (echo, us/call): op direct unix mcp unix-vs-mcp resolve 1.4 42.9 4809 112x read_bytes(16) 0.8 78.6 4450 57x read_bytes(4096) 117.1 144.4 6312 44x disasm_line 2.9 81.2 5423 67x xrefs_to 40.6 77.9 4770 61x decompile(cached) 2907 2712 47456 18x Takeaways: * A lean local IPC round-trip is ~40-80us — ~60-110x cheaper than the mcp HTTP/JSON path (~5ms/call floor), while KEEPING crash isolation and the main-thread decoupling (the freeze/segfault costs of full in-process). * Bulk bytes are the tell: read_bytes(4096) is 144us unix vs 117us direct (1.2x overhead) but 6.3ms over mcp — pickle ships 4096 raw bytes; mcp hex-encodes + JSON-wraps them. The hex view would feel instant on unix. * ~50us/call = ~20k calls/sec vs mcp's ~200/sec: most of idatui's prefetch/ paging/caching machinery exists to hide the 5ms; on a unix worker you'd barely need it. Conclusion this run supports: the sweet spot is Option C (own thin worker), not full in-process — you capture ~99% of the practical latency win without the UI freeze during analysis or the loss of crash isolation.

experiments: in-process idalib vs mcp-transport spike (throwaway)

2026-07-23T23:09:51Z

Standalone, not wired into the app. A tiny Backend seam (functions/resolve/ read_bytes/disasm_line/decompile/xrefs_to) with two impls — DirectBackend (import idapro, in-process) and McpBackend (the current HTTP/JSON tool calls) — so the "keep the transport or go direct?" question is measurable and feelable. --bench : A/B latency table (opens a copy in-process; also hits :8745 if up) --tui : minimal Textual app on the in-process backend; F5 decompiles INLINE so you feel the main-thread hitch, 'd' decompiles all (big freeze) Findings (echo, this box), all reproducible: * open+auto-analysis in-process: ~0.4s (the whole "loading" cost, on the main thread). * per-op latency, direct vs mcp: resolve 2.0us 4755us 2392x read_bytes(16) 1.2us 4657us 3845x read_bytes(4096) 112us 6037us 54x disasm_line 2.9us 5239us 1805x xrefs_to 25us 4962us 200x decompile(cached) 2.8ms 51ms 18x i.e. the mcp transport has a ~5ms/call floor regardless of op; the fast ops idatui spams while scrolling are 1000-4000x cheaper in-process (which is why the prefetch/paging/caching machinery exists). * hard constraints proven separately: idalib must be imported/opened on the MAIN python thread (installs a SIGINT handler) and every call must be on it ("Function can be called from the main thread only"); execute_sync from a worker thread HANGS (no UI pump in headless). So in-process, IDA owns the one main thread and blocks the event loop for each call — fine at <1ms, a hitch at decompile (~150ms cold), a freeze during analysis. That main-thread coupling, not just crash isolation, is what the subprocess boundary buys.