| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
step 1
First concrete step off the mcp HTTP transport. Instead of reimplementing ~25
tools, reuse ida-pro-mcp's tool *functions* verbatim and replace only the
transport + process management:
* idatui/worker.py — opens ONE database in-process on the main thread (as idalib
requires), imports ida_pro_mcp (which registers every stock + our patched-in
custom tool against MCP_SERVER), then serves MCP_SERVER.tools.methods[name]
(**args) over a unix socket with length-prefixed pickle. Serial on the main
thread (idalib is single-threaded; tools run inline through execute_sync).
Session-management tools (idb_open/idb_save/server_health/idb_list) are shimmed
since the worker *is* the single session.
* idatui/worker_client.py — WorkerClient exposes the exact surface the app/domain
use on the client (call/call_envelope/connect/set_db/resolve_db/list_sessions/
health/keepalive/close) and returns byte-identical payloads (the worker calls
the same functions IDAClient.call ultimately hits). So domain.py and the app
are UNCHANGED — you just construct a WorkerClient instead of an IDAClient.
Calls are serialized under a lock over one socket; keepalive is a no-op (the
worker is ours and never idles out).
Not wired into the app yet — the mcp path is fully intact.
Verified without idalib: pickle framing round-trips arbitrary payloads incl raw
bytes; WorkerClient has full IDAClient surface; call_envelope produces the
result.structuredContent shape domain.decompile() reads. The idalib E2E
(experiments/worker_smoke.py drives the real domain.Program read path through the
worker) is written but couldn't run here — this sandbox has degraded to reaping
any idalib spawn; the underlying unix-socket protocol already ran clean in the
inproc_spike bench (~50us/call), and the worker dispatches the same tool
functions the HTTP path does, so shapes match by construction.
Next: stand up progress reporting during analysis, then flip _connect/_reconnect
to build a WorkerClient behind a flag and run the pilot suite against it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds UnixWorkerBackend (Option C): the same DirectBackend, but in a child
process that opens idalib on ITS main thread and serves one client serially over
an AF_UNIX socket with length-prefixed pickle (bytes ride raw — no hex, no JSON).
bench() is now generic over {direct, unix, mcp}; --worker runs the child.
3-way result (echo, us/call):
op direct unix mcp unix-vs-mcp
resolve 1.4 42.9 4809 112x
read_bytes(16) 0.8 78.6 4450 57x
read_bytes(4096) 117.1 144.4 6312 44x
disasm_line 2.9 81.2 5423 67x
xrefs_to 40.6 77.9 4770 61x
decompile(cached) 2907 2712 47456 18x
Takeaways:
* A lean local IPC round-trip is ~40-80us — ~60-110x cheaper than the mcp
HTTP/JSON path (~5ms/call floor), while KEEPING crash isolation and the
main-thread decoupling (the freeze/segfault costs of full in-process).
* Bulk bytes are the tell: read_bytes(4096) is 144us unix vs 117us direct (1.2x
overhead) but 6.3ms over mcp — pickle ships 4096 raw bytes; mcp hex-encodes +
JSON-wraps them. The hex view would feel instant on unix.
* ~50us/call = ~20k calls/sec vs mcp's ~200/sec: most of idatui's prefetch/
paging/caching machinery exists to hide the 5ms; on a unix worker you'd barely
need it.
Conclusion this run supports: the sweet spot is Option C (own thin worker), not
full in-process — you capture ~99% of the practical latency win without the UI
freeze during analysis or the loss of crash isolation.
|
|
|
Standalone, not wired into the app. A tiny Backend seam (functions/resolve/
read_bytes/disasm_line/decompile/xrefs_to) with two impls — DirectBackend
(import idapro, in-process) and McpBackend (the current HTTP/JSON tool calls) —
so the "keep the transport or go direct?" question is measurable and feelable.
--bench : A/B latency table (opens a copy in-process; also hits :8745 if up)
--tui : minimal Textual app on the in-process backend; F5 decompiles INLINE
so you feel the main-thread hitch, 'd' decompiles all (big freeze)
Findings (echo, this box), all reproducible:
* open+auto-analysis in-process: ~0.4s (the whole "loading" cost, on the main
thread).
* per-op latency, direct vs mcp:
resolve 2.0us 4755us 2392x
read_bytes(16) 1.2us 4657us 3845x
read_bytes(4096) 112us 6037us 54x
disasm_line 2.9us 5239us 1805x
xrefs_to 25us 4962us 200x
decompile(cached) 2.8ms 51ms 18x
i.e. the mcp transport has a ~5ms/call floor regardless of op; the fast ops
idatui spams while scrolling are 1000-4000x cheaper in-process (which is why
the prefetch/paging/caching machinery exists).
* hard constraints proven separately: idalib must be imported/opened on the MAIN
python thread (installs a SIGINT handler) and every call must be on it
("Function can be called from the main thread only"); execute_sync from a
worker thread HANGS (no UI pump in headless). So in-process, IDA owns the one
main thread and blocks the event loop for each call — fine at <1ms, a hitch at
decompile (~150ms cold), a freeze during analysis. That main-thread coupling,
not just crash isolation, is what the subprocess boundary buys.
|
