diff options
| author | user <user@clank> | 2026-08-07 15:14:30 +0200 |
|---|---|---|
| committer | user <user@clank> | 2026-08-07 15:14:30 +0200 |
| commit | 72fce7da1a1fd6527e389ffeb0f951157523589a (patch) | |
| tree | 3f08a83f0d99f3d3fc7069b7be00d235bab82817 /docs/CODEMODE_PORT.md | |
| parent | Stop tracking 157MB of core dumps, and ignore them (diff) | |
| parent | docs: upstream findings for the ida-codemode maintainers (diff) | |
| download | ida-tui-72fce7da1a1fd6527e389ffeb0f951157523589a.tar.gz ida-tui-72fce7da1a1fd6527e389ffeb0f951157523589a.tar.xz ida-tui-72fce7da1a1fd6527e389ffeb0f951157523589a.zip | |
Merge the IDA Code Mode port
Replaces the private idalib worker (idatui/worker.py + worker_client.py, with
server/patch_server.py injecting tools into ida-pro-mcp) with an ordinary
client of ida_codemode.client.DatabaseHandle. A database open by an IDA GUI is
reused; otherwise Code Mode starts or shares a managed idalib worker. The TUI
no longer owns an IDA process, and closing it releases only its lease.
Based on Duncan Ogilvie's port, rebased onto ~150 commits of local work it
predated. The rebase itself was mechanical; landing it was not. Nine defects
had to be fixed before the feature set was whole again, none of which the
patch's own tests could catch:
- DatabaseHandle.open() takes image_base, not loading_address: every
connect() would have raised TypeError on the first call
- five operations our tree had grown were simply missing (flowchart, so the
graph view was dead; op_format/pc_nums/pc_num_format, so 'o'/'O' were;
survey_binary)
- set_comments wrote only the disassembly comment, so comments never
appeared in pseudocode
- xref_query returned rows in raw IDA order, and 'follow the call' silently
followed the fall-through instead
- rename accepted one edit per category, so bulk symbol import was dead
- decompile ran decomp_map's full per-column ctree sweep to fill in a
per-line address anchor
- heads shipped without operand extents or the digest protocol
- the package became unimportable without ida_codemode installed, which
killed the offline test suites
Verified against the pre-codemode tag rather than against assumptions: the
full suite is 788 passed / 0 failed, and the pilot's 301 checks match the old
backend exactly. Performance is within 2x on the listing hot path and faster
on decompile, disasm and connect, after fixing two runtime costs that are
documented for upstream in docs/CODEMODE_UPSTREAM.md.
Test runtime came down from ~9m20s to 115s along the way -- not by removing
checks, but by removing four kinds of waiting-on-a-guess that were also
hiding real failures.
Diffstat (limited to 'docs/CODEMODE_PORT.md')
| -rw-r--r-- | docs/CODEMODE_PORT.md | 175 |
1 files changed, 175 insertions, 0 deletions
diff --git a/docs/CODEMODE_PORT.md b/docs/CODEMODE_PORT.md new file mode 100644 index 0000000..4b4bc45 --- /dev/null +++ b/docs/CODEMODE_PORT.md @@ -0,0 +1,175 @@ +# ida-tui → IDA Code Mode port + +This port is an experiment: can ida-tui be implemented as an ordinary client of +`ida_codemode`, sharing GUI databases and managed idalib workers instead of +owning a private worker and depending on ida-pro-mcp tool functions? + +## Result + +Yes for the database lifecycle and the complete current TUI feature set, with a +small number of operations implemented using IDAPython inside Code Mode's +`execute_python` sandbox because ida-domain does not yet expose the required +behavior. + +The old components are gone: + +- `idatui/worker.py` (private pickle/socket idalib process) +- `idatui/worker_client.py` +- `server/patch_server.py` (ida-pro-mcp tool injection) + +The replacement is `idatui/codemode_client.py`. + +## Lifecycle mapping + +`CodeModeClient.connect()` calls `ida_codemode.client.DatabaseHandle.open()`. +Resolution is therefore Code Mode's resolution, not ida-tui's: + +1. Match a registered GUI by executable path. +2. Otherwise match the owner of the expected IDB. +3. Otherwise serialize creation and start a managed `ida-codemode-worker`. +4. Establish an authenticated SSE lease. +5. Wait through the public autoanalysis route. + +The handle's registry entry supplies the backend, PID, executable path, IDB path, +and record ID used by the status/pool layers. + +Closing ida-tui closes only its lease. It never closes a GUI or kills an idalib +process. A managed worker saves and exits under Code Mode's own policy after its +last lease disappears. A second agent or TUI can keep using the same instance. + +This also changes project pooling semantics. `DatabasePool` is an LRU pool of +leases, not process ownership. Managed-IDB save-on-evict remains; budget eviction +does not implicitly save a GUI. Eviction cannot force a shared worker to exit, +and GUI process memory is only advisory. + +## ida-domain coverage + +The remote snippets receive Code Mode's preloaded `db` (`ida_domain.Database`). +The following TUI needs map to public ida-domain entities: + +| TUI need | ida-domain surface | +|---|---| +| Function paging, lookup, names, sizes | `db.functions` | +| Segments and names | `db.segments` | +| Instructions and plain disassembly | `db.instructions`, `db.functions.get_instructions()` | +| Heads and item classification | `db.heads`, `db.bytes` | +| Bytes and strings | `db.bytes`, `db.strings` | +| Symbol resolution and rename | `db.names`, `db.functions` | +| Comments | `db.comments` | +| Imports and exports | `db.imports`, `db.entries` | +| Xrefs and fine type predicates | `db.xrefs` / `XrefInfo` | +| Named types, members, parse/apply | `db.types` | +| Function prototypes and local variables | `db.pseudocode`, `PseudocodeFunction.local_variables` | +| Decompilation text and object references | `db.pseudocode` | + +All values are reduced to JSON primitives inside the database process. No SWIG +or ida-domain object crosses the Code Mode boundary. + +## Remaining IDAPython gaps + +Code Mode intentionally allows regular Python imports, so these features still +work, but they identify useful additions to ida-domain: + +1. **Rich continuous listing** + - ida-domain enumerates defined heads and renders plain disassembly. + - ida-tui also needs coalesced undefined runs, IDA colour-tag spans, function + banners, code-label rows, file-region offsets, and expanded struct members. + - The `heads` operation uses `ida_bytes`, `ida_lines`, and related modules for + this presentation model. + +2. **Instruction/function carving** + - Creating an instruction and walking a speculative decode run requires + `ida_ua.create_insn` and processor flow/return checks. + - Function creation exists in ida-domain; the explicit-end fallback still + needs lower-level item boundaries. + +3. **ARM/Thumb state** + - T-register ranges and segment addressing use `ida_segregs`, `ida_idp`, and + `ida_segment`. There is no equivalent ida-domain operation. + +4. **Detailed decompiler diagnostics and line maps** + - Pseudocode text, ctree objects, and the address map are available through + ida-domain. + - Reproducing IDA's per-rendered-line coverage uses + `cfunc.get_line_item`; obtaining the exact Hex-Rays failure description + uses `hexrays_failure_t`. + +5. **A few type/item primitives** + - Deleting a named local type and some exact item-undefinition/data-creation + behavior still use `ida_typeinf`/`ida_bytes` directly. + +These uses are isolated in `idatui/codemode_client.py`; the paging and Textual +layers do not import IDAPython. + +## API limitations exposed by the port + +### No rollback or close-without-save + +A Code Mode lease has no rollback operation. Closing a GUI handle leaves the GUI +state as-is. A managed idalib worker currently saves when its final lease closes. +Consequently ida-tui's old “discard & quit” guarantee cannot be implemented. +The UI now labels this choice “leave as-is & quit” and does not explicitly save, +but managed-worker policy may still persist the changes. + +A true discard action would need a Code Mode/database API for transaction-like +rollback, a close policy on a newly-owned worker, or a TUI-managed disposable DB +copy. + +### Typed loader options only + +`DatabaseHandle.open()` supports processor, natural loading address, file type, +output database, and fresh-database selection. It does not support ida-tui's +arbitrary `ida_args` escape hatch. The adapter rejects unsupported switches +rather than silently loading at the wrong architecture/base. + +### No database-change notification stream + +The lease reports liveness, not mutations. If a GUI user or another Code Mode +client renames/retypes content while ida-tui is open, already-materialized TUI +caches are not invalidated automatically. TUI-originated edits invalidate their +own caches correctly. A database revision counter or change feed would make +shared interactive editing robust. + +### Discovery requires a path for ambiguity + +`ida-tui` with no path attaches automatically when exactly one database is +registered. With several registrations it lists them and requires an explicit +executable/IDB path. There is not yet a pre-connection database picker in the +Textual UI. + +### `DatabaseHandle` import stability + +The usable library primitive currently lives at +`ida_codemode.client.DatabaseHandle`; `ida_codemode.__init__` exports nothing. +The port therefore depends on a submodule path. Exporting the handle and public +client exceptions from the package root would make the supported library API +clearer. + +## Safety differences + +ida-tui no longer removes `.id0/.id1/.id2/.nam/.til` files before opening. That +was only defensible when the TUI exclusively owned a private process; it is +unsafe when a GUI or another client may own the database. Code Mode registry +locks, health probes, and IDA itself now arbitrate ownership. + +The old pane “reap private workers” behavior is obsolete. A TUI crash closes its +lease at the socket/kernel boundary; Code Mode decides whether a managed worker +still has clients and when it should stop. + +## Verification surfaces + +The non-IDA suite verifies project staging, LRU lease behavior, load-option +translation, and adapter response/error normalization. The existing live suites +remain the end-to-end contract: + +```sh +uv run python tests/test_codemode_client.py +uv run python tests/test_pool.py +uv run python tests/test_project.py +uv run python tests/test_scenarios.py /path/to/binary +``` + +For GUI reuse, open the same binary in an IDA with the Code Mode plugin, confirm +it appears in `ida_codemode.registry.discover_instances()`, then launch +`ida-tui /path/to/binary`. The TUI status/`CodeModeClient.backend` should report +`gui`, and closing the TUI must leave IDA open. |
