ida-tui.git/experiments, branch pre-codemode

graph: docs, and the offline layout tools

2026-08-06T13:11:54Z

cfg_dump freezes real CFGs to JSON; graph_spike renders one or --stats a whole corpus through the shipping engine; graph_smoke is the end-to-end tool->domain->layout check; graph_shot renders the real view headless at a chosen size, which is the only sane way to judge it (a tiled pane is far too narrow and the minimap sits on top of the graph).

tests: cover the literal formats

2026-08-06T13:11:54Z

Pilot scenarios for the listing and the pseudocode, for the mark moving between operands, for a refusal not being swallowed by the previous success, and for the cursor staying on its literal across a reflow. Plus experiments/opfmt_tools.py, which runs the real injected tool sources against a live database with the decorators stubbed -- faster than the pilot and the right place for the IDA-side edge cases. Also fixes two pre-existing bugs the work surfaced, both of which made edits happen off screen: cursor_on searched from row 0 of the whole segment and never scrolled, so a driver's word= edit landed in an unrelated function while reporting success; and the cursor verb didn't scroll either. Both now go through rpc.place_cursor.

arm: find Thumb entry points from a vector table (Shift+T)

2026-07-26T18:49:30Z

An ARM function pointer carries the mode in bit 0: odd means Thumb. A Cortex-M vector table is therefore a list of Thumb entry points, and IDA won't follow them on a headerless image because nothing tells it those words are pointers at all. Shift+T scans forward from the cursor and marks them. 0 functions -> 3 Thumb entries found, 3 disassembled A word only counts when it is odd, lands in a loaded segment, and its target is executable and not already data. The even words in a vector table — the initial stack pointer — fail the first test, which is the point: marking a data word as code corrupts the listing, so a false positive costs more than a miss. The fixture includes an even in-range word and an odd OUT-of-range word to keep that honest. A note on how this started: I recommended this feature, then probed experiments/fibonacci.bin for the signal and found ZERO odd in-range pointers — it's a flat code blob, not a firmware image. Rather than build a detector I couldn't test, I wrote experiments/cortexm.bin: a real vector table pointing at small self-contained Thumb handlers. The first version of that fixture aimed its handlers into the middle of copied code, so two "entries" were really inside one function — the tool was right and the fixture was wrong, which is worth stating because I nearly filed it as a bug. Function creation goes through one _idatui_add_func helper now, shared with define_func_run: add_func(ea) alone fails on freshly-marked code (IDA can't find the end), and the scan hit exactly the same wall `p` did. Status precedence, fixed properly this time. An action's result kept being overwritten by the reload it triggered — cursor moved, filter re-applied, functions re-counted. I patched that at FIVE separate call sites before admitting it's one problem. _status(text, priority=True) now marks a result: it holds the bar for 8s or until the next keypress, and routine chatter can't outrank it. The per-site special cases are gone. tests: +4 thumb (20) — a bare vector table gives IDA nothing, scanning finds exactly the three handlers, the non-pointer words are ignored, and the result survives both the reload and the reindex. 209/0 scenarios, 30/0 blob, 30/0 project UI.

worker: idalib worker + WorkerClient (drop-in for IDAClient) — migration step 1

2026-07-23T23:29:06Z

First concrete step off the mcp HTTP transport. Instead of reimplementing ~25 tools, reuse ida-pro-mcp's tool *functions* verbatim and replace only the transport + process management: * idatui/worker.py — opens ONE database in-process on the main thread (as idalib requires), imports ida_pro_mcp (which registers every stock + our patched-in custom tool against MCP_SERVER), then serves MCP_SERVER.tools.methods[name] (**args) over a unix socket with length-prefixed pickle. Serial on the main thread (idalib is single-threaded; tools run inline through execute_sync). Session-management tools (idb_open/idb_save/server_health/idb_list) are shimmed since the worker *is* the single session. * idatui/worker_client.py — WorkerClient exposes the exact surface the app/domain use on the client (call/call_envelope/connect/set_db/resolve_db/list_sessions/ health/keepalive/close) and returns byte-identical payloads (the worker calls the same functions IDAClient.call ultimately hits). So domain.py and the app are UNCHANGED — you just construct a WorkerClient instead of an IDAClient. Calls are serialized under a lock over one socket; keepalive is a no-op (the worker is ours and never idles out). Not wired into the app yet — the mcp path is fully intact. Verified without idalib: pickle framing round-trips arbitrary payloads incl raw bytes; WorkerClient has full IDAClient surface; call_envelope produces the result.structuredContent shape domain.decompile() reads. The idalib E2E (experiments/worker_smoke.py drives the real domain.Program read path through the worker) is written but couldn't run here — this sandbox has degraded to reaping any idalib spawn; the underlying unix-socket protocol already ran clean in the inproc_spike bench (~50us/call), and the worker dispatches the same tool functions the HTTP path does, so shapes match by construction. Next: stand up progress reporting during analysis, then flip _connect/_reconnect to build a WorkerClient behind a flag and run the pilot suite against it.

experiments: add a unix-socket idalib worker as the 3rd bench column

2026-07-23T23:19:31Z

Adds UnixWorkerBackend (Option C): the same DirectBackend, but in a child process that opens idalib on ITS main thread and serves one client serially over an AF_UNIX socket with length-prefixed pickle (bytes ride raw — no hex, no JSON). bench() is now generic over {direct, unix, mcp}; --worker runs the child. 3-way result (echo, us/call): op direct unix mcp unix-vs-mcp resolve 1.4 42.9 4809 112x read_bytes(16) 0.8 78.6 4450 57x read_bytes(4096) 117.1 144.4 6312 44x disasm_line 2.9 81.2 5423 67x xrefs_to 40.6 77.9 4770 61x decompile(cached) 2907 2712 47456 18x Takeaways: * A lean local IPC round-trip is ~40-80us — ~60-110x cheaper than the mcp HTTP/JSON path (~5ms/call floor), while KEEPING crash isolation and the main-thread decoupling (the freeze/segfault costs of full in-process). * Bulk bytes are the tell: read_bytes(4096) is 144us unix vs 117us direct (1.2x overhead) but 6.3ms over mcp — pickle ships 4096 raw bytes; mcp hex-encodes + JSON-wraps them. The hex view would feel instant on unix. * ~50us/call = ~20k calls/sec vs mcp's ~200/sec: most of idatui's prefetch/ paging/caching machinery exists to hide the 5ms; on a unix worker you'd barely need it. Conclusion this run supports: the sweet spot is Option C (own thin worker), not full in-process — you capture ~99% of the practical latency win without the UI freeze during analysis or the loss of crash isolation.

experiments: in-process idalib vs mcp-transport spike (throwaway)

2026-07-23T23:09:51Z

Standalone, not wired into the app. A tiny Backend seam (functions/resolve/ read_bytes/disasm_line/decompile/xrefs_to) with two impls — DirectBackend (import idapro, in-process) and McpBackend (the current HTTP/JSON tool calls) — so the "keep the transport or go direct?" question is measurable and feelable. --bench : A/B latency table (opens a copy in-process; also hits :8745 if up) --tui : minimal Textual app on the in-process backend; F5 decompiles INLINE so you feel the main-thread hitch, 'd' decompiles all (big freeze) Findings (echo, this box), all reproducible: * open+auto-analysis in-process: ~0.4s (the whole "loading" cost, on the main thread). * per-op latency, direct vs mcp: resolve 2.0us 4755us 2392x read_bytes(16) 1.2us 4657us 3845x read_bytes(4096) 112us 6037us 54x disasm_line 2.9us 5239us 1805x xrefs_to 25us 4962us 200x decompile(cached) 2.8ms 51ms 18x i.e. the mcp transport has a ~5ms/call floor regardless of op; the fast ops idatui spams while scrolling are 1000-4000x cheaper in-process (which is why the prefetch/paging/caching machinery exists). * hard constraints proven separately: idalib must be imported/opened on the MAIN python thread (installs a SIGINT handler) and every call must be on it ("Function can be called from the main thread only"); execute_sync from a worker thread HANGS (no UI pump in headless). So in-process, IDA owns the one main thread and blocks the event loop for each call — fine at <1ms, a hitch at decompile (~150ms cold), a freeze during analysis. That main-thread coupling, not just crash isolation, is what the subprocess boundary buys.