diff options
| -rw-r--r-- | .agents/skills/idatui/SKILL.md | 408 | ||||
| -rw-r--r-- | README.md | 18 |
2 files changed, 425 insertions, 1 deletions
diff --git a/.agents/skills/idatui/SKILL.md b/.agents/skills/idatui/SKILL.md new file mode 100644 index 0000000..336b31a --- /dev/null +++ b/.agents/skills/idatui/SKILL.md @@ -0,0 +1,408 @@ +--- +name: idatui +description: Build/extend "idatui", a keyboard+mouse TUI frontend for IDA Pro (Textual over IDA Code Mode / ida-codemode). Use when working in this repository — adding views/features, fixing the listing/pseudocode/hex/graph panes, function list, search, follow/xrefs, rename/retype, projects, traces, or navigation/history. Captures the project layout, how to run/test, and the hard-won Code Mode + Textual gotchas that are expensive to rediscover. +compatibility: "Backend is the ida-codemode LIBRARY (a normal dependency; see pyproject.toml). Needs a licensed IDA Pro and a venv with idapro + textual + ida_codemode — this project assumes ~/ida-venv. Pure test suites must keep running under a stdlib-only python3. No private worker, no server, no ida-pro-mcp." +--- + +<!-- This is the in-repo copy of the idatui agent skill, discovered automatically + from .agents/skills/ by harnesses implementing https://agentskills.io. + Paths below are relative to the repository root unless stated otherwise. --> + +# idatui — IDA TUI frontend + +A minimal, tasteful, keyboard-first (mouse-capable) TUI for IDA Pro, driving +**IDA Code Mode** (`ida_codemode.client.DatabaseHandle`) — it takes a *lease* on a +database that is either an already-open IDA **GUI** session or a shared **managed +idalib worker**. idatui never owns or terminates an IDA process. Repo: +this repository. Built with **Textual**. This skill is the memory of how it +is built and the traps that cost real time to find. + +> **History, so old notes don't mislead you.** idatui used to run its own private +> idalib worker (`idatui/worker.py` + `worker_client.py`) over a unix socket, +> reusing **ida-pro-mcp**'s `@tool` functions, with custom tools injected into the +> installed package by `server/patch_server.py`. **All of that is deleted.** If you +> see it referenced anywhere, the reference is stale. The IDAPython that used to +> live in those tools now lives in `idatui/remote_tools.py` and runs inside the +> Code Mode sandbox. + +## Run & test + +```bash +cd <repo root> +./ida-tui /abs/path/to/binary # lease a matching GUI session, else a managed worker +./ida-tui # attach when exactly one database is registered +./ida-tui fw.bin --processor arm --base 0x8000000 # raw/headless binaries +./ida-tui bin --rpc /tmp/ida.sock # + puppeteer socket (docs/RPC.md) +./ida-tui bin --trace trace.0.log # + Tenet execution trace +./ida-tui --project t.idatui-project # multi-binary project (docs/PROJECTS.md) +# other flags: --ttl, --no-keepalive, --ida-args. +# idatui/launch.py is the logic; `ida-tui`, `python -m idatui.launch` and +# `python -m idatui` all resolve to it. + +# spawn+drive from another pane (agent-driven RE) — see the `idatui-rpc` skill +~/ida-venv/bin/python -m idatui.pane spawn --open /abs/bin # prints {sock,...} +python3 -m idatui.drive where # terse, auto-finds the socket +python3 -m idatui.drive pc main strrchr # pseudocode of main, grepped +python3 -m idatui.drive rename sub_5BE0 stdout_isatty # goto+rename in one +python3 -m idatui.rpcclient --sock <sock> keys g m a i n enter # raw JSON transport +``` + +### The test front door is `tests/run.py` + +Every suite is a standalone script with its own tally (`N passed, M failed`) — +**this repo does not use pytest.** Each test file declares `NEEDS_IDA = True/False` +and `run.py` reads that marker with `ast`; a file with no marker is a hard error. + +```bash +python3 tests/run.py --fast # every PURE suite, 380 checks, ~0.8s, any python3 +python3 tests/run.py --list # what would run, and whether it needs IDA +python3 tests/run.py graph -x # substring select, fail-fast +~/ida-venv/bin/python tests/run.py # FULL gate: 19 files, 1031 checks, ~49s, serial on purpose +``` + +**Deliberately serial** — do not re-add `--jobs`. 4 concurrent IDA suites took the +suite 153s → 296s and killed three with broken-pipe worker failures; idalib +contends hard enough that extra processes only starve each other, and a starved +worker gets reaped mid-analysis, which reads as a flaky test rather than as load. + +Narrow iteration on the big pilot suite (166s full, so don't): + +```bash +~/ida-venv/bin/python tests/test_scenarios.py targets/echo --only rename # ~2s +~/ida-venv/bin/python tests/test_scenarios.py --list # scenario names +~/ida-venv/bin/python tests/test_scenarios.py targets/echo --profile # where time went +``` + +`--profile` reports per-scenario settle/wait/keystroke seconds and — the important +one — any wait that **EXPIRED**, with its line number. An expired wait costs its +whole timeout *and* means the check after it passed vacuously. Don't optimise this +suite by guessing; fix the top line. + +Offline/no-IDA tools (any python3, ~1s): `python3 tests/test_graph.py` (+ a real +128-function corpus as an argument). Smoke tests through the real backend: +`PYTHONPATH=. ~/ida-venv/bin/python experiments/worker_smoke.py` (want `VERDICT: OK`), +`graph_smoke.py`, `graph_shot.py sub_61D0 170 46` (LOOK at the graph). + +Commit style: small, one concern per commit, run the pilot suite first. + +## The two interpreters + +| python | has | use for | +|---|---|---| +| `python3` (system) | stdlib only | every **pure** suite. No IDA, no Code Mode. | +| `~/ida-venv/bin/python` | textual + idapro + ida_codemode | anything `ida`, and the TUI | + +(`~/ida-venv` is this project's assumed location for the IDA-capable venv — see the +README. Substitute your own; the split between the two is what matters.) + +A `pure` file must keep running under system `python3` — house rule, and it is why +`codemode_client.py` defers its `ida_codemode` import instead of doing it at module +top. The repo's own `.venv` also has idapro+textual+ida_codemode and works, but +`~/ida-venv` is the documented runner. + +**`experiments/` scripts need `PYTHONPATH=.`** — idatui is installed editable in +`.venv`, not in `~/ida-venv`, so running an experiment under `~/ida-venv` without +it fails with `ModuleNotFoundError: idatui`. + +## Architecture (keep the layers separate) + +- **`idatui/codemode_client.py`** — the backend adapter. Turns idatui's small, + address-centric domain operations into self-contained Python snippets and runs + them through Code Mode's one broad operation, `execute_python`. `_OPERATIONS` is + the snippet table; `_script()` binds JSON args without interpolating user text + into code; `_PACK_EPILOGUE`/`_unpack` carry the answer as one pre-serialised JSON + string. Snippets prefer the public **ida-domain** `db` object. Shared error types + + `Session` live in `idatui/errors.py`. +- **`idatui/remote_tools.py`** (1.5k lines) — the IDAPython that runs *inside* the + Code Mode sandbox, for the things ida-domain has no surface for (IDA-coloured + listing rows, creating instructions, ARM T-state, Hex-Rays line maps/failures). + Carried over verbatim from the tools idatui was developed against. It is + installed into the database process on first call and re-installed if that + process restarts (`_NEED_LIB` handshake). +- **`idatui/pool.py`** — `DatabasePool`, LRU **leases** over Code Mode databases for + a project. Owns *client interest*, never an IDA process. +- **`idatui/domain.py`** — paging/caching over the client. `FunctionIndex` (lazy, + clamped, filter globs), `DisasmModel` (block-cached windowed disasm + prefetch + + ea→index), `decompile`, `xrefs_from/to`, `function_of`, `resolve`. Synchronous + + thread-safe; the UI runs it in workers. +- **`idatui/app.py`** (8.2k lines) — the Textual app. `ListingView`/`DecompView`/ + `HexView`/`GraphView` are line-**virtualized `ScrollView`s** (render only the + viewport). Shared mixins: `ColumnCursor` (h/l/w/b, mouse, paging, `_apply_scroll`), + `SearchMixin` (`/` `?` incremental), `NavMixin` (enter=follow, x=xrefs, n=rename). + Modals: `XrefsScreen`, `SymbolPalette`, `SearchPalette`, `StringsPalette`, + `StructEditor`, `LoadOptionsScreen`, `ProjectPalette`, `RegWriteScreen`, … + (Note: the listing view is `ListingView`, not `DisasmView` — old notes lie.) +- **Controllers split out of `app.py`** — `edit_ctl.py` (everything that WRITES: + rename/comment/retype/define, and the cache invalidation each needs), + `trace_ctl.py` (trace navigation behind `--trace`), `prompt.py` (the six one-line + bottom prompts, one implementation), `search.py` (what did Ctrl+F mean: text vs + bytes), `diag.py` (where swallowed errors go — ~50 `except Exception` sites). +- **`idatui/project.py` + `index.py`** — multi-binary projects (a JSON file plus a + sidecar dir) and one on-disk searchable index across every binary, so search works + for binaries whose worker isn't running. See `docs/PROJECTS.md`. +- **`idatui/journal.py` + `findings.py`** — a record of the edits idatui makes, kept + inside the database, and the markdown session report built from it. The point: the + output of an RE session is what you *learned*, and the `.i64` cannot tell you which + names were yours and which were IDA's. +- **`idatui/graph.py`** — the CFG **graph view**'s layout engine (`space` toggles it). + Pure python: no IDA, no Textual, no I/O, so `tests/test_graph.py` runs offline in ms. + Textbook Sugiyama: break cycles → longest-path layer → dummy nodes → + median/transposition ordering → priority x-coords → port+channel routing. Sizing is + INJECTED (`layout(blocks, sizer)`). Returns a `Layout` whose `Painting` is an INDEX + (per-row h-runs, bucketed v-run intervals, point marks), never a canvas — a + 424-block function is ~13M cells. Node bodies are the SAME `Head` rows the listing + renders, so highlighting/renames/xrefs/trail work inside boxes for free. Backend = + one operation, `flowchart(addr)`. Full doc: `docs/GRAPH_VIEW.md`. +- **`idatui/graph_triskel.py`** — a SECOND layout engine (SESE decomposition) behind + `graph.layout(engine=...)`, preferred by `auto` up to 250 blocks, off a local fork + of triskel. Optional: without it, layout falls back to the pure engine. NOT a + dependency (PyPI's pytriskel has no wheel for current pythons and a binding bug); + install our fork's bindings instead (`uv pip install <triskel-checkout>/bindings/python`; + `pyproject.toml` records the path this project uses). See `docs/TRISKEL_EVAL.md`. +- **`idatui/trace.py`** — Tenet execution trace reader (~400 lines, ours, not a port). + Parses the delta log, indexes by IP for trail painting, reconstructs register/memory + state at any timestamp. Differential-tested against Tenet's reference reader. +- **`idatui/highlight.py`** — Pygments C lexer → Rich styles (Textual has NO cpp + grammar). Two consumers, one palette: `highlight_c` (Segments, read-only pseudocode) + and `CTextArea` (the struct editor's EDITABLE C). Spans there are BYTE offsets per + line, and the `TextAreaTheme` must set only `syntax_styles` — a `base_style` would + override the widget's CSS background. +- **`idatui/formats.py`** — recognising what a file is, and what to ask when IDA's + loaders don't match (processor/base/entry for raw blobs). +- **`idatui/_sync.py`** — shared "UI has settled" logic (`wait_for`/`drain`/`settle`), + used by BOTH the pilot tests (`Ctx.wait`) and the live RPC driver. One source of truth. +- **`idatui/rpc.py`** — optional unix-socket server to puppeteer the live TUI: raw + `keys`/`text`; structured reads `state`/`view`/`screen`/`functions`/`pseudocode`/ + `disassembly`/`xrefs_to`/`xrefs_from`/`resolve`; semantic verbs `goto`/`rename`/ + `comment`/`retype`/`follow`/`back`/`toggle_view`/`hex`/`xrefs`/`symbols`/`structs`/ + `search`/`opfmt`/`select`/`save`/`move`/`cursor`/`trace`. Started from `on_mount` + with `--rpc <sock>`; handlers run on the app loop. No auth (0600 socket). + `rpcclient.py` = stdlib raw JSON client+CLI; `drive.py` = the ergonomic layer + (prefer it); `pane.py` spawns/stops TUI panes in tmux/zellij. Full reference: + `docs/RPC.md`; workflow: the `idatui-rpc` skill. Semantic verbs type through the + real prompts with a per-char delay (livestream aesthetic); `move`/`cursor` are fast. + +## IDA Code Mode gotchas + +- **`ida-codemode` is a normal dependency** (`pyproject.toml` pins `>=0.3.1`); these + notes are current as of **0.3.2**. If you install it **editable from a git checkout** + (handy for tracking upstream), remember that a `git pull` in that checkout swaps the + backend under the TUI **immediately, with no reinstall** — convenient, but it means an + upstream change can alter behaviour without anything in this repo changing. Run the + full gate after any such pull. +- **Two big client-side workarounds existed and are now DELETED, because 0.3.2 fixed + them upstream.** Don't re-add them, and don't trust older docs that describe them: + - the runtime used to wrap every `execute_python` in `sys.settrace(timeout_trace)`, + and that trace returned *itself* → LINE tracing in every frame → `ida_bytes.get_flags` + 5.49µs vs 0.106µs native (**52x**). 0.3.2 has zero `settrace`; the deadline is a + C-level thread interrupt (`runtime._interrupt_thread`). Our strip measured **0.99x**. + - `to_jsonable()` walked every returned object (66ms on a 200-row page, 114x + `json.dumps`). 0.3.2's `serialization.dumps_json` fast-paths to the C encoder. + Our `_PACK_EPILOGUE` measured **0.97x** and is kept only to pin encoder settings + (compact separators, `default=str`) — no longer load-bearing for performance. + - Re-measure both with `PYTHONPATH=. ~/ida-venv/bin/python experiments/bench_pack_trace.py`. +- **`docs/CODEMODE_UPSTREAM.md` is our findings report to the ida-codemode maintainers, + and every item is re-checked against 0.3.2.** Items **1, 2 and 3 are FIXED** (trace + hook, to_jsonable, the execute_sync floor). Items **4–9 are still open** — loader + switches fatal on reopen, replaced IDB under a live lease, no close-without-save, no + change notification, package exports, no `py.typed`/handle Protocol — and they are + open *by construction*: `client.py`, `registry.py`, `resolver.py`, `server.py`, + `database.py` and `worker.py` are byte-identical between 0.3.1 and 0.3.2, and those + items all live in those files. Read it before assuming a Code Mode behaviour is a bug + in our code. **Cheap way to re-check after any upstream pull:** + `for f in client registry resolver server database worker; do git diff --quiet OLD..HEAD -- ida_codemode/$f.py; done` + — if they're all unchanged, items 4–9 cannot have moved. +- **The old ~2ms per-operation floor is GONE in 0.3.2 (7.0x).** Same-box A/B: + `execute_python("result = 1")` was **2.055ms** on 0.3.1 and is **0.294ms** on 0.3.2, + about the cost of a bare HTTP GET — the `execute_sync` marshalling that was ~93% of + the floor is effectively gone. The old rule "never make a call per row" is much + weaker now. It is still not free, and the existing batched design is still right — + **don't rewrite working code to be chatty** — but measure before assuming the floor + forbids a design. Call volume today: opening a listing is 1 call, scrolling 2000 rows + is 8, a 1060-block graph is 4. +- **Leases, not ownership.** Closing the client releases only its lease. Killing a pilot + leaves a Code Mode worker for its `--lease-grace` (20s); it self-exits and holds only + that run's temp `.i64`, so it does not block a new run. +- **Never `kill -9` an IDA/worker process.** IDA unpacks a `.i64` to + `foo.id0/.id1/.id2/.nam/.til` while open; a clean close re-packs and deletes them, a + `-9` leaves them and the `.i64` then fails to reopen ("Failed to open database"). Fix: + remove those stale files (NEVER the `.i64` — it is the real DB, and nothing is saved + unless you called save). Quit the TUI cleanly (`q`). +- **A stale registry entry looks like a hard failure**: `instance NNN-xxxx owns + /path/foo.i64 but is unavailable: Connection refused` just means a previous process + exited without releasing. Wait out the grace period rather than deleting databases. +- **idalib is CPU/IO-heavy and reap-prone in loaded sandboxes.** Under heavy load the + open + auto-analysis can exceed a command timeout and get SIGKILLed — looks like a + hang or empty output (worse through a `| grep`, whose buffer is lost on kill), but it + is CPU starvation, not a code bug. Check `uptime`; verify on an idle box. +- **Response shape** (preserved from the old backend, so `domain.py` was unchanged by + the port): `{"result": [ { "data": [...], "next_offset": <int|null> } ]}`. + **Paginate by `offset += len(data)`**, never by `next_offset`. +- **`docs/PAGING_FINDINGS.md` measurements predate Code Mode** and say so: the silent + per-call caps (700 / 500 / collapse-to-10) were *ida-pro-mcp server* limits. The port + keeps the conservative page sizes, but enumeration now runs through ida-domain in our + own snippets, so those caps are historical, not Code Mode constraints. + +## Textual pitfalls (the ones that actually bit us) + +Fuller notes in `docs/TEXTUAL_NOTES.md`. + +- **`BINDINGS` only merge from `DOMNode` subclasses.** A plain mixin's `BINDINGS` are + silently dropped — list them explicitly on each view (e.g. `*SearchMixin.SEARCH_BINDINGS`). +- **`ScrollView` + `render_line`**: `y` is the SCREEN line; add `self.scroll_offset.y` + yourself. `watch_scroll_y` only repaints when the **rounded** scroll changes. +- **Setting `virtual_size` then an immediate `scroll_to` CLAMPS to 0** (max_scroll_y isn't + recomputed until layout). Apply the scroll now AND again via `call_after_refresh` with + `self.refresh(layout=True)`. Do NOT zero `virtual_size` on load (snaps scroll to 0 → + flash). See `ColumnCursor._apply_scroll`. +- **Pilot lays out SYNCHRONOUSLY**, so programmatic scroll always has a valid range there + — it MASKS the real-terminal clamp/no-repaint bug. Test the **render** (trace + `render_line`'s `scroll_offset` at paint), not just `scroll_offset.y`. +- **`widget.loading = True` covers via `_cover_widget`** (not a normal child; `query` + won't find it) and **blurs focus** (focus→None) so `Tab` falls back to focus-nav. Make + critical bindings `priority=True`; restore focus when done. +- **Mouse**: `event.get_content_offset(widget)` gives the offset past padding/border; add + `scroll_offset` for the virtual (line,col). `event.chain>=2` = double-click. Account + for any left gutter (`_col_offset`). +- **Cheap cursor moves**: `refresh(Region(0,row,w,1))` for just the changed rows, and + `reactive(x, repaint=False)` to avoid an implicit full repaint. +- **`Input` defaults to a 3-row bordered widget**; for a 1-row prompt use `border:none` + and DON'T `dock:bottom` it alongside the `Footer` (same row; the Footer paints over it). + +## Key patterns already implemented + +- **Name-generation cache invalidation** (`Program._name_gen`/`bump_names`): a rename + bumps the gen; disasm block caches are cleared (disasm names are live); `decompile` is + gen-checked and force-recompiled lazily on mismatch. Call `bump_names()` in `_after_rename`. +- **Cursor+scroll history**: `NavEntry` stores `cursor, cursor_x, scroll_y` (listing) and + `dec_cursor, dec_cursor_x, dec_scroll_y, dec_scroll_x` (pseudocode). `_save_current_pos()` + snapshots BOTH views right before a push; restore on `_open_entry`/`show`. +- **Follow**: name-based (refs → `resolve`) THEN **address-based fallback** (parse + `/*0xEA*/` → `xrefs_from` → first code target). The address path is immune to a stale + name right after a rename. +- **`include_addresses`** appends `/*0xEA*/` per pseudocode line — the only per-line + address anchor; use it for address-based follow. +- **Operand formats** (`o`/`O`): the listing uses IDA's operand types + (`ida_bytes.op_hex/op_dec/...`), pseudocode uses Hex-Rays' own per-(ea,opnum) + numforms — a SEPARATE set; the listing's format does not reach it. The cycle ring only + offers stops that change what you see. `B` cycles the opcode-bytes column. + +## Testing gotchas + +- **Scenario suite** (`tests/test_scenarios.py`): independent `@scenario` fns over one + shared boot; a `Ctx` wraps app+pilot. `reset()` baselines between them (default + `_pref=decomp`). Set up view context with `c.open(ea, "decomp"|"disasm")`, not + `goto_ui`+`tab`: with `_pref=decomp` the goto opens in decomp and `tab` flips you to + disasm, so a follow keystroke hits the wrong pane and the jump silently never happens. +- **`goto` to a non-existent function is a no-op** → "jump back" tests then pass + trivially. Always navigate to a REAL second function. +- **Scroll-restore tests with the cursor at the viewport top (`rel=0`) are degenerate** + (scroll-into-view derives the same scroll). Put the cursor MID-viewport (`rel>0`). +- **Cold session = slow first analysis** → `wait_until` timeouts → flaky "want 0"/cursor 0 + failures. Re-run on a warm session before believing a regression. +- **Edits mutate the `.i64`.** Always revert renames (via API) so tests are idempotent, + and use a PID-unique temp name to avoid collisions from a prior crashed run. +- **The pilot suite is not the whole story.** Two backend bugs the Code Mode port shipped + were invisible to it: `test_rawimage_rpc.py` owns **batch** `rename_many` + define/blob + + opfmt over RPC (where a list-vs-dict bug hid), `test_blob_ui.py` the raw-image UI, + `test_project_ui.py` multi-binary projects, `test_trace_*` the Tenet integration. If + you touch a backend operation, grep for its callers and run the suite that owns them. +- **Simulated keypresses cost 85ms each unless you patch Textual.** `Pilot.press` → + `App._press_keys` calls `wait_for_idle` twice per key, which sleeps in 20ms granules + until process time stops advancing. `tests/_fixtures.py: fast_keys()` replaces it + (every UI suite calls it at import): `wait_for_idle` → `await asyncio.sleep(0)`, and + `Pilot.press` → send keys then **`settle(app)`**. The second half is the point — + deleting the heuristic without it broke nine checks. +- **A settled app has not necessarily been laid out or painted.** Worker-driven → + `settle(app, pred)`; **timer**-driven (the function filter's 0.08s debounce) → wait on + the effect; **frame**-driven (`widget.region`, `size`, a repaint trace) → wait on the + geometry. Never a longer sleep. + +## Graph-view gotchas (`idatui/graph.py`, `GraphView`) — see `docs/GRAPH_VIEW.md` + +- **A self-loop deadlocks Kahn ranking.** A block that jumps to itself never drains its + own in-degree, so ranking stalls and everything downstream stays at rank 0 — the graph + collapses to ~3 layers and comes out absurdly WIDE (280 cols for 8 blocks). Drop + self-loops from the layout graph, draw them as a `↺` marker. `_assign_ranks` also + force-releases the most-constrained survivor when the queue drains early. +- **Crossing minimisation is the whole runtime.** Naive transposition recounts crossings + globally per candidate swap = O(n³): 20.4 SECONDS on a 424-block function. Fenwick-tree + inversion counting + a local `O(deg(a)*deg(b))` swap delta → 152ms (corpus 21s → 224ms). +- **Dummy nodes are why no edge crosses a box.** A long edge occupies real horizontal + space as a chain of dummies, so routing never has to cut through a node. + `tests/test_graph.py` asserts 0 such cells over a 128-function corpus — keep that test, + it is the invariant the whole design rests on. +- **Horizontal runs live in a channel BELOW A WHOLE LAYER**, never at a per-node y offset. + Per-node offsets look fine until a layer holds boxes of different heights, then edges + saw through the taller neighbour. +- **Everything downstream trusts `app._active`.** Adding the `"graph"` mode meant auditing + every `_active ==` / `_active in` switch; the one that was missed (`_active_code_view()` + returning None) crashed the app the first time a prompt closed in graph mode. + `grep -n '_active ==\|_active in' idatui/app.py idatui/rpc.py` before adding another mode. +- **A stale async view-load must not steal the view.** `_apply_graph` returns early unless + graph mode is still wanted (`_graph_sticky or _active == "graph"`). +- **A `ScrollView` paints its scrollbar over the last column**, which ate the minimap's + right border — it is inset two columns for that reason. +- **Judge the rendering with `experiments/graph_shot.py`** (headless pilot at a size you + choose), not by looking at the pane you're in — a tiled pane is far too narrow and the + minimap will sit on top of the graph. +- Offline tools: `experiments/cfg_dump.py` (freeze real CFGs to JSON), + `graph_spike.py` (render/`--stats` a corpus), `graph_compare.py`, `graph_smoke.py`. + +## Terminal graphics (`idatui/kittygfx.py`) — the startup splash + +- **Never sniff `$TERM`/`$KITTY_WINDOW_ID`/`$COLORTERM` for protocol support.** Under a + multiplexer that passes kitty graphics through (zellij did, in the case that taught us + this), TERM is `xterm-256color` and every one of those vars is EMPTY while the + protocol answers `OK`. + Ask the terminal: 1x1 graphics query + DA1, DA1 is the sync point. +- **THE BIG ONE: query and upload go on OPPOSITE sides of the alternate screen.** The + detection query must run BEFORE Textual starts (it reads stdin on its own thread and + eats the reply) — but the IMAGE must be uploaded AFTER Textual switched to the alt + screen. An image uploaded on the primary screen **cannot be placed from the alternate + one: the placement returns OK and draws nothing.** Hence `supported()` in `launch.py`, + `upload()` in `LoadingScreen.on_mount`. +- **Unicode placeholders (`U=1` + U+10EEEE) are NOT supported here**, though plain + `a=p`/`a=T` placement works. That is why `textual-image` renders nothing: its whole TGP + path is placeholders, and its only fallback is sixel — advertised by DA1, also broken. +- **Direct placement is anchored to screen cells; Textual does not know it exists.** Place + after layout (`call_after_refresh`), re-anchor when the region repaints (throttled), and + DELETE on unmount, or the image sits on top of the disassembly forever. +- Ruled OUT, don't re-chase: an opaque cell background does NOT hide the image, and + z-index is not needed (z omitted / z=1 / z=-1 all render over a painted background). +- **Correct escapes != visible pixels.** Every automated check can only prove what we + SENT. Bisect "renders nothing" by reproducing the exact sequence in a RAW pane (no + Textual), one variable at a time. `$IDATUI_KITTY=0/1` forces detection, + `$IDATUI_KITTY_LOG=/tmp/x.log` traces every decision. +- **Only ONE test pane on screen at a time**, or you will attribute a working image to + the wrong pane and "confirm" a broken build. + +## Multiplexer safety when driving panes + +Relevant to any agent (or human) driving the TUI from a second pane. + +- **`zellij action close-pane` closes the FOCUSED pane** — which, when an agent runs it, + is normally the agent's own. Always target explicitly: + `zellij action close-pane --pane-id terminal_N`, or better + `python -m idatui.pane stop --pane <id>`. (Learned the hard way.) +- **Never `pkill -f <pattern>` here.** Your own shell's command line contains the + pattern you are matching, so `pkill -f "tests/run.py"` kills the very shell issuing it + and the rest of the command silently never runs. Kill by **pid** (`pgrep -af` first). +- **Backgrounding a long run needs all three fds redirected**, not just `&`. A + backgrounded subshell still holds the inherited stdout, so a caller that waits for EOF + on that pipe (many agent harnesses, and `$(...)`) blocks anyway even though the job is + "in the background". Use `setsid CMD </dev/null >LOG 2>&1 &`. +- **Don't pipe a TUI's stdout through `tee`** when testing tty-dependent behaviour; + `isatty()` goes False and terminal detection silently disables itself. Log via stderr. +- **`zellij action new-pane` does NOT inherit the caller's environment**, so + `FOO=1 ... pane spawn` never reaches the TUI. Put it in the argv: + `new-pane -- env FOO=1 <python> -m idatui.launch ...`. + +## Where the backlog lives + +`TODO` in the repo root is the live list (it is long and current). Open items from the +Code Mode port: run the full pilot suite against **both** GUI and managed backends, add +database revision/change notifications for cross-client cache invalidation, and decide +how "discard changes" should work (Code Mode's final workers save). +Docs: `CODEMODE_UPSTREAM.md`, `GRAPH_VIEW.md`, `PAGING_FINDINGS.md`, `PROJECTS.md`, +`RPC.md`, `SPLIT_VIEW.md`, `TEXTUAL_NOTES.md`, `TRISKEL_EVAL.md`. @@ -175,7 +175,7 @@ test is empty while the protocol works fine). ```sh python3 tests/run.py --fast # 380 checks, <1s, any python3 — between edits python3 tests/run.py --list # what runs, and what needs IDA -python3 tests/run.py # 788 checks, ~2m — before a commit +python3 tests/run.py # 1031 checks, ~50s — before a commit ``` Every suite declares `NEEDS_IDA`; `--fast` runs only the pure ones (stdlib, no @@ -194,6 +194,22 @@ flake, and the four ways a test here wastes minutes. - [`docs/RPC.md`](docs/RPC.md) — the RPC protocol, verb by verb - [`docs/GRAPH_VIEW.md`](docs/GRAPH_VIEW.md) — how the graph is laid out +- [`docs/CODEMODE_UPSTREAM.md`](docs/CODEMODE_UPSTREAM.md) — findings from porting to + IDA Code Mode, and which are fixed upstream +- [`docs/PROJECTS.md`](docs/PROJECTS.md), [`docs/SPLIT_VIEW.md`](docs/SPLIT_VIEW.md), + [`docs/TEXTUAL_NOTES.md`](docs/TEXTUAL_NOTES.md), + [`docs/PAGING_FINDINGS.md`](docs/PAGING_FINDINGS.md), + [`docs/TRISKEL_EVAL.md`](docs/TRISKEL_EVAL.md) + +### Working on this with an LLM agent + +[`.agents/skills/idatui/SKILL.md`](.agents/skills/idatui/SKILL.md) is an +[Agent Skills](https://agentskills.io/specification) skill describing the +architecture, the run/test loop and the traps that cost real time here (Code Mode, +Textual, the graph engine, terminal graphics). Harnesses implementing that standard +discover `.agents/skills/` automatically; others can be pointed at the file directly. +A user-level skill of the same name takes precedence, so delete or symlink yours if +you keep a personal copy. — |