| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Mechanical part of the port: the 27-file patch was cut against a base ~148
commits behind us, so it did not apply. Resolved 11 conflicts (all of them
diff drift, not semantic clashes) and the three file deletions:
- app.py: the patch re-inserted _do_rename/_do_name_addr/_seek_split etc. as
"theirs" because our tree moved them to edit_ctl.py/trace_ctl.py. Kept ours
and applied the real intent (WorkerClient->CodeModeClient, .call->.invoke,
_open_worker_client->_open_database_client) at their current homes.
- domain.py: kept Head as a NamedTuple -- the patch reverted it to a frozen
dataclass, which the perf work measured at 2.9us vs 1.9us per row on a
quarter-million-row walk. Dropped _fetch_output (no download_url under Code
Mode) and its now-dead urllib/json imports.
- pane.py: the patch's deletion swallowed our zellij support along with the
worker-reaping block it meant to remove. Kept zellij, removed the reaping.
- test_scenarios.py: the idb_save->save_database teardown hunk belongs to
tests/_fixtures.py in our tree; applied it there and kept our pc_num_format
scenario that the drift landed on.
Three defects in the patch itself, fixed here:
- It made "import idatui" hard-require ida_codemode, so every offline suite
died at import -- including the pure ones (graph/index/trace) that are the
house rule for "tests/run.py --fast". The import is now deferred and gated
on the binding, which is also what lets the port's own contract tests
inject a fake DatabaseHandle.
- project.stage() inlined an ida_codemode.registry import and treated "library
not installed" as "someone owns this database", which broke IDA-free project
staging. Ownership lookup moved to codemode_client.database_owner().
- tests/test_codemode_client.py had no NEEDS_IDA marker, which tests/run.py
rejects outright.
Offline suite: 301 passed, 0 failed. Against master's 344 the whole delta is
accounted for: -40 worker_client (module deleted), -18 launch sweep checks
(behaviour deliberately removed) +3 guarding that it stays removed, +2 pool
(GUI-save semantics), +13 new codemode_client contract tests.
NOT yet done, and the port is not functional without it: the adapter is
missing five operations our tree grew since the patch's base (flowchart,
op_format, pc_nums, pc_num_format, survey_binary) and its "heads" predates
back-walking and digest/expect.
|
| |
|
|
|
|
|
|
| |
cfg_dump freezes real CFGs to JSON; graph_spike renders one or --stats a
whole corpus through the shipping engine; graph_smoke is the end-to-end
tool->domain->layout check; graph_shot renders the real view headless at a
chosen size, which is the only sane way to judge it (a tiled pane is far
too narrow and the minimap sits on top of the graph).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Pilot scenarios for the listing and the pseudocode, for the mark moving
between operands, for a refusal not being swallowed by the previous
success, and for the cursor staying on its literal across a reflow. Plus
experiments/opfmt_tools.py, which runs the real injected tool sources
against a live database with the decorators stubbed -- faster than the
pilot and the right place for the IDA-side edge cases.
Also fixes two pre-existing bugs the work surfaced, both of which made
edits happen off screen: cursor_on searched from row 0 of the whole
segment and never scrolled, so a driver's word= edit landed in an
unrelated function while reporting success; and the cursor verb didn't
scroll either. Both now go through rpc.place_cursor.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An ARM function pointer carries the mode in bit 0: odd means Thumb. A Cortex-M
vector table is therefore a list of Thumb entry points, and IDA won't follow them
on a headerless image because nothing tells it those words are pointers at all.
Shift+T scans forward from the cursor and marks them.
0 functions -> 3 Thumb entries found, 3 disassembled
A word only counts when it is odd, lands in a loaded segment, and its target is
executable and not already data. The even words in a vector table — the initial
stack pointer — fail the first test, which is the point: marking a data word as
code corrupts the listing, so a false positive costs more than a miss. The
fixture includes an even in-range word and an odd OUT-of-range word to keep that
honest.
A note on how this started: I recommended this feature, then probed
experiments/fibonacci.bin for the signal and found ZERO odd in-range pointers —
it's a flat code blob, not a firmware image. Rather than build a detector I
couldn't test, I wrote experiments/cortexm.bin: a real vector table pointing at
small self-contained Thumb handlers. The first version of that fixture aimed its
handlers into the middle of copied code, so two "entries" were really inside one
function — the tool was right and the fixture was wrong, which is worth stating
because I nearly filed it as a bug.
Function creation goes through one _idatui_add_func helper now, shared with
define_func_run: add_func(ea) alone fails on freshly-marked code (IDA can't find
the end), and the scan hit exactly the same wall `p` did.
Status precedence, fixed properly this time. An action's result kept being
overwritten by the reload it triggered — cursor moved, filter re-applied,
functions re-counted. I patched that at FIVE separate call sites before
admitting it's one problem. _status(text, priority=True) now marks a result: it
holds the bar for 8s or until the next keypress, and routine chatter can't
outrank it. The per-site special cases are gone.
tests: +4 thumb (20) — a bare vector table gives IDA nothing, scanning finds
exactly the three handlers, the non-pointer words are ignored, and the result
survives both the reload and the reindex. 209/0 scenarios, 30/0 blob, 30/0
project UI.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
step 1
First concrete step off the mcp HTTP transport. Instead of reimplementing ~25
tools, reuse ida-pro-mcp's tool *functions* verbatim and replace only the
transport + process management:
* idatui/worker.py — opens ONE database in-process on the main thread (as idalib
requires), imports ida_pro_mcp (which registers every stock + our patched-in
custom tool against MCP_SERVER), then serves MCP_SERVER.tools.methods[name]
(**args) over a unix socket with length-prefixed pickle. Serial on the main
thread (idalib is single-threaded; tools run inline through execute_sync).
Session-management tools (idb_open/idb_save/server_health/idb_list) are shimmed
since the worker *is* the single session.
* idatui/worker_client.py — WorkerClient exposes the exact surface the app/domain
use on the client (call/call_envelope/connect/set_db/resolve_db/list_sessions/
health/keepalive/close) and returns byte-identical payloads (the worker calls
the same functions IDAClient.call ultimately hits). So domain.py and the app
are UNCHANGED — you just construct a WorkerClient instead of an IDAClient.
Calls are serialized under a lock over one socket; keepalive is a no-op (the
worker is ours and never idles out).
Not wired into the app yet — the mcp path is fully intact.
Verified without idalib: pickle framing round-trips arbitrary payloads incl raw
bytes; WorkerClient has full IDAClient surface; call_envelope produces the
result.structuredContent shape domain.decompile() reads. The idalib E2E
(experiments/worker_smoke.py drives the real domain.Program read path through the
worker) is written but couldn't run here — this sandbox has degraded to reaping
any idalib spawn; the underlying unix-socket protocol already ran clean in the
inproc_spike bench (~50us/call), and the worker dispatches the same tool
functions the HTTP path does, so shapes match by construction.
Next: stand up progress reporting during analysis, then flip _connect/_reconnect
to build a WorkerClient behind a flag and run the pilot suite against it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds UnixWorkerBackend (Option C): the same DirectBackend, but in a child
process that opens idalib on ITS main thread and serves one client serially over
an AF_UNIX socket with length-prefixed pickle (bytes ride raw — no hex, no JSON).
bench() is now generic over {direct, unix, mcp}; --worker runs the child.
3-way result (echo, us/call):
op direct unix mcp unix-vs-mcp
resolve 1.4 42.9 4809 112x
read_bytes(16) 0.8 78.6 4450 57x
read_bytes(4096) 117.1 144.4 6312 44x
disasm_line 2.9 81.2 5423 67x
xrefs_to 40.6 77.9 4770 61x
decompile(cached) 2907 2712 47456 18x
Takeaways:
* A lean local IPC round-trip is ~40-80us — ~60-110x cheaper than the mcp
HTTP/JSON path (~5ms/call floor), while KEEPING crash isolation and the
main-thread decoupling (the freeze/segfault costs of full in-process).
* Bulk bytes are the tell: read_bytes(4096) is 144us unix vs 117us direct (1.2x
overhead) but 6.3ms over mcp — pickle ships 4096 raw bytes; mcp hex-encodes +
JSON-wraps them. The hex view would feel instant on unix.
* ~50us/call = ~20k calls/sec vs mcp's ~200/sec: most of idatui's prefetch/
paging/caching machinery exists to hide the 5ms; on a unix worker you'd barely
need it.
Conclusion this run supports: the sweet spot is Option C (own thin worker), not
full in-process — you capture ~99% of the practical latency win without the UI
freeze during analysis or the loss of crash isolation.
|
|
|
Standalone, not wired into the app. A tiny Backend seam (functions/resolve/
read_bytes/disasm_line/decompile/xrefs_to) with two impls — DirectBackend
(import idapro, in-process) and McpBackend (the current HTTP/JSON tool calls) —
so the "keep the transport or go direct?" question is measurable and feelable.
--bench : A/B latency table (opens a copy in-process; also hits :8745 if up)
--tui : minimal Textual app on the in-process backend; F5 decompiles INLINE
so you feel the main-thread hitch, 'd' decompiles all (big freeze)
Findings (echo, this box), all reproducible:
* open+auto-analysis in-process: ~0.4s (the whole "loading" cost, on the main
thread).
* per-op latency, direct vs mcp:
resolve 2.0us 4755us 2392x
read_bytes(16) 1.2us 4657us 3845x
read_bytes(4096) 112us 6037us 54x
disasm_line 2.9us 5239us 1805x
xrefs_to 25us 4962us 200x
decompile(cached) 2.8ms 51ms 18x
i.e. the mcp transport has a ~5ms/call floor regardless of op; the fast ops
idatui spams while scrolling are 1000-4000x cheaper in-process (which is why
the prefetch/paging/caching machinery exists).
* hard constraints proven separately: idalib must be imported/opened on the MAIN
python thread (installs a SIGINT handler) and every call must be on it
("Function can be called from the main thread only"); execute_sync from a
worker thread HANGS (no UI pump in headless). So in-process, IDA owns the one
main thread and blocks the event loop for each call — fine at <1ms, a hitch at
decompile (~150ms cold), a freeze during analysis. That main-thread coupling,
not just crash isolation, is what the subprocess boundary buys.
|
