aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* tests: repair the two seams the ida-nexus port left behindblasty7 days3-12/+89
| | | | | | | | | | | | | | | Two scenario monkeypatch sites still hooked client.invoke -- the rename to call() updated the call sites but not the patches, so reprime_is_free and split_view crashed instead of counting. call() takes the remote_ops declaration itself now, so both count by its __name__. remote_ops imported RemoteModule inside _bindings(), which made test_nexus_client (NEEDS_IDA = False) unrunnable under a stdlib-only python3 -- the house rule tests/run.py --fast depends on. The import moves to the same eagerly-if-present, bound-to-None-so-patchable contract nexus_client uses, and the test injects FakeRemoteModule/FakeRemoteError exactly like its other fakes: real names when the library is installed, strict fakes when it is not.
* Merge PR #1 from mrexodia: Windows support and auto-refresh on eventsblasty7 days42-3645/+6770
|\ | | | | | | | | | | | | | | | | | | | | https://github.com/blasty/ida-tui/pull/1 — from the ida-nexus (né ida-codemode) maintainer: kitty-graphics fallback for platforms without termios, Ctrl+R view refresh, typed remote operations through ida_nexus RemoteModule, live auto-refresh from other clients' IDB events (upstream item 7), discard-without-save (item 6), and the ida-nexus 0.7.0 rename. tests/test_kittygfx.py is the union of both sides' files: our escape- construction checks plus the PR's cross-platform fallback checks.
| * Switch to ida-nexusDuncan Ogilvie7 days38-244/+243
| |
| * Adopt idb_events and remote module features from ida-codemodeDuncan Ogilvie8 days16-2982/+5833
| |
| * Add Ctrl+R to refresh all viewsDuncan Ogilvie11 days4-2/+206
| |
| * Add kitty graphics fallback for Windows supportDuncan Ogilvie11 days2-3/+105
| |
* | splash: give the logo a placement id so re-anchoring replaces, not stacksblasty7 days4-6/+278
|/ | | | | | | | | | | | | | | | A kitty placement is identified by (image id, placement id); an a=p with no p= key is anonymous and every one stacks another copy. The splash re-anchors on every progress note (~5/s), so a 60s load ended with ~300 placements of an RGBA logo alpha-compositing over each other -- soft edges creeping to solid, and the terminal re-rendering all of them per frame. Same pair every time (LOGO_PLACEMENT) = the terminal REPLACES the placement, so re-anchoring is free and atomic. That is also why on_resize no longer clear()s first (that showed a hole for a frame), and why a shrunk-to-nothing region now drops the placement instead of leaving a stale one anchored. tests/test_kittygfx.py pins the escapes (pure, stdlib); experiments/splash_place_count.py counts what a real splash sends.
* docs: be honest that the triskel fork is unpublishedblasty13 days4-18/+18
| | | | | | | | | The install instructions pointed at ~/dev/triskel, a path only I have, so a reader could not act on them. Say plainly that the patched pytriskel is not released yet and that native is the supported default. Also drops the .fastfeedback/SPEED.md references and refreshes a stale branch name and commit hash in bench_ops.py.
* docs: drop the triskel evaluation artifactsblasty13 days5-354/+1
| | | | | | | | | TRISKEL_EVAL.md and graph_compare.py were intermediary working docs from choosing the SESE engine. Triskel now ships alongside the native layout engine, so the evaluation no longer needs to be in the tree; GRAPH_VIEW.md keeps the part that describes shipping behaviour. Also drops the local-only ~/dev/triskel/PATCHES.md pointer from pyproject.
* README: drop the ASCII mock and the LLM-agent sectionblasty13 days1-19/+0
| | | | | | The mock terminal is a poor stand-in for a real demo; a gif/screenshots should go there instead. The agent section pointed at .agents/, which is no longer part of the repo.
* Add MIT licence and contributor guideblasty13 days3-0/+102
| | | | | Also ignore the local-only working artifacts (.auto/, .agents/, plan/, rehearsed-engineer.md) so they stay on disk and out of the repo.
* Re-check the upstream findings report against 0.6.1blasty2026-08-141-36/+79
| | | | | | | | | | | | | | | | | | | Items 4, 8 and 9 are fixed (loader switches on reopen, package exports, py.typed), so the report and the skill no longer claim 4-9 are all open. Item 7 is partial: DatabaseManager grew a DatabaseEventCallback, but there is still nothing an external caching client can observe, which was the point. 5 and 6 are untouched. The "are these files byte-identical" re-check recipe is removed: it named client.py/registry.py/resolver.py, which 0.5.x renamed, so it would have reported "unchanged" forever. The replacement is a diff of __all__ and the DatabaseOpenOptions fields, both of which the contract suite now asserts whenever the library is importable. Also records two things that cost time here: the loader-switch strip needs input_path != source, so an .i64 passed directly alongside load options still fatals; and an editable checkout does not enforce its own dependency floors, so a `git pull` to 0.6.1 leaves ida-domain/zeromcp silently too old.
* Port to the ida-codemode 0.5.3+ public APIblasty2026-08-147-454/+154
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 0.5.x turned ida_codemode.__all__ into a real public API and hid the rest: client.py -> handle.py, registry.py -> _registry.py + instances.py, resolver.py -> _resolver.py. Every import we had was from a module that no longer exists, so the TUI could not attach at all. - handle.entry -> handle.instance (RegistryEntry -> DatabaseInstance) - 30 keyword-only options -> DatabaseOpenOptions(...) passed as options= - IdbBusy -> DatabaseBusyError - InstanceDisconnected/ClientError -> DatabaseDisconnected/CodeModeConnectionError - our scan_instances()+idb_key() ownership walks -> find_database_owner() - our FileLock poking (_wait_for_entry_release) -> wait_database_released() - registry.discover_instances() -> discover_databases() + InstanceState _database_exists() is deleted with it: upstream now drops the loader switches itself when reopening an existing IDB (_resolver._build_worker_command), which is the same fix we had client-side. See docs/CODEMODE_UPSTREAM.md section 4 for the one invocation that still slips through. The offline contract suite has to keep running under a stdlib-only python3, where every Code Mode name is bound to None -- so it now injects a strict fake DatabaseOpenOptions and a real DatabaseBusyError exception alongside the fake handle. Without the latter, `except DatabaseBusyError` is `except None`, and the TypeError it raises masks whatever actually failed inside the try. The loader-option names moved inside the options dataclass, so the guard that caught `loading_address` vs `image_base` moved with them (_option_fields_are_real). uv.lock pins 0.6.1; ~/ida-venv and .venv are on 0.6.1 with the ida-domain 0.5.1 / zeromcp 1.8.0 floors it requires. Full gate: 1065 passed.
* Fix: rebuilding the row index on every switch back to the listingblasty2026-08-102-0/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reported as "decompiler -> listing feels slow", and it was: ~900ms per Tab. The listing view re-primes every time it is shown, and priming now builds the whole row index. build_from_index() had no idempotence, so each switch back re-ran segment_index over the entire segment. Measured on bash, going back to the listing away from the primed viewport: press Tab: 904ms -> 9.7ms, and 5 backend calls -> 0 The index is a pure function of the database and the model is thrown away and rebuilt whenever anything moves the walk (stale_structure), so a model that is already indexed can return immediately. Nothing caught this because the listing was CORRECT the whole time -- only slow. Every structural assertion passed, boot still measured fast, and the suite has no notion of "how many calls did that keystroke cost". The new scenario counts backend calls across three view switches and asserts segment_index is not among them; with the guard removed again it fails. Latent and NOT fixed here: materialising a page the viewport reaches for the first time still happens inside render_line, i.e. an RPC (~20ms) on the UI loop. That predates this change -- it is how skeleton pages have always worked -- and is small enough not to read as a stall, but it is the same shape of bug and wants prefetching onto the worker that already exists for pages. Full gate: 1064 passed.
* Build the listing index in one call: boot 9.3s -> 1.25s, 911 calls -> 4blasty2026-08-105-5/+244
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The listing used to learn its shape by fetching it. Even after skeleton pages that was 457 round trips and 227k rows for a 1.2MB bash, to end up knowing how many rows there are and where each one is. segment_index(detail=True) now returns exactly that -- every row's address, kind and size as packed arrays, plus the page boundaries -- from one walk that builds no rows and renders no text. ListingModel.build_from_index() decodes it straight into _heads/_head_eas/_row_at/_by_ea/_page_*, marks every row _SKELETON_GEN, and declares itself complete. _grow has nothing left to stream. Nothing else in the model changed, because a row without text is a state it already had: the FIRST read of a page materialises it through the same _ensure_text/_ensure_page path a rename uses. That is why this is a ~90 line change to a core view rather than a rewrite. bash boot: 911 calls / 9.26s -> 4 calls / 1.25s 7.4x whole census (boot + 9 UI actions): 933 calls -> 30 Two things had to be exactly right, and both are tested rather than argued: * the ROW COUNT, or the scrollbar lies. Verified equal to a fully streamed model, and every row's ea/kind/size equal too, 228,659 of them, zero mismatches. * the PAGE BOUNDARIES, or _ensure_page refetches a page that does not line up, fails its structure check and triggers a full rebuild. heads() pages on PHYSICAL rows; anchoring every N LOGICAL rows looks identical (the two only diverge once a segment holds an undefined run) and would have been a lurking bug on .bss. Anchors now carry [logical_row, ea, head_index] taken at the real boundary, and are asserted equal to the streamer's own. Transport note: the packed arrays are base64, not raw bytes. _PACK_EPILOGUE serialises with json.dumps(default=str), which turns bytes into their repr -- 2.97MB arrived as 11.26MB of unparseable text before that was spotted. The new test builds both models back to back and compares every internal array. An earlier version compared against the app's long-lived model and was off by one row, because scenarios before it rename and define things: that model describes the database at boot, not now. Full gate: 1063 passed, twice.
* segment_index: the row total in one call instead of 458blasty2026-08-103-0/+143
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The listing streams a whole segment for one reason -- to know how many rows it has, so the scrollbar and paging are right. Even as skeletons that is 458 round trips and 227k rows for a 1.2MB bash, none of which is displayed. segment_index walks the same items and counts what heads() WOULD emit, building none of them, and returns the total plus [row, ea] anchors every 500 rows. Measured on bash, same process and database: segment_index : 228,659 rows, 1 call, 501ms streaming : 228,659 rows, 458 calls, 1836ms 3.7x Exactness is the whole point, so it mirrors _rows_for's arithmetic rather than approximating it: 3 banner rows at a function start, a label row for a named code head that is not one, the head row, struct members for data, 2 footer rows at a function end, and an undefined run counted as its byte length because the client presents one collapsed row as that many logical rows. A count that is off by a handful means the scrollbar lies and a seek lands on the wrong row, so the test compares against a fully streamed model in the same process rather than against a tolerance, and checks that every anchor names the address of the row it claims. Getting that right took a false alarm worth recording: the count first looked 35 rows short of a model built by the pilot, which turned out to be a DIFFERENT DATABASE (tests run on a pristine scratch copy). Against the same database it matches exactly, head for head, with zero differing addresses. Nothing consumes this yet. Spending it means teaching ListingModel to hold sparse pages seeked through the anchors instead of one dense array grown from the segment start, which is a real change to the core view and wants its own run at it. Full gate: 1054 passed.
* domain: build Head positionally, and drop a dead test from _as_intblasty2026-08-102-9/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Head is the most-constructed object in the codebase -- 227k of them to stream one bash -- and from_raw was paying for two things it did not need. _as_int ended in a ternary whose arms were BOTH int(v, 16), so the isinstance+startswith in front of them decided nothing and ran on every address the client parses (55k times per 60 pages). Removed; the function is now the isinstance it always was. Head(...) was built by keyword, which makes the tuple match names against fields; positional is the same object with none of that. Address conversion is inlined for the same reason from_raw exists at all. Together 30% off Head construction (2.27 -> 1.59ms per 2003 rows, 0.34us a row), with output verified identical against the previous implementation over both skeleton and full rows. isinstance was kept over the 5%-faster "type(v) is int" because the two differ for bool and int subclasses, and that is not a trade worth making for 5% of 12% of a page. End to end this is ~2.5% of bash's boot (3117 -> 3065ms): Head construction was ~12% of the client half, and the client half is ~54% of what is left. Adds experiments/profile_client.py -- profile_remote.py's counterpart for the half of a page load that happens outside the database process, which is where the remaining time now is. Full gate: 1050 passed.
* remote_tools: item class by mask instead of is_code/is_data callsblasty2026-08-101-3/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The listing walk asked ida_bytes.is_code/is_data ~6 times per head (the row's kind, plus twice more via _is_unknown_f from _advance and _rows_for). Each is a python wrapper around a C call; the mask compare is the same question with no call at all. Equivalence was proven, not assumed: checked against is_code/is_data over all four classes x 16,020 synthetic flag values, and over every distinct flag in a real database (125 across 207,191 heads). Zero mismatches. Output is byte-identical over 3003 rows spanning all five row kinds. (The digest DOES differ run to run, which looked alarming until it turned out the same unmodified code differs too: hash() is seed-randomised per process and the digest is documented as per-process. Cross-process digest comparison proves nothing.) Worth 5.6% on a skeleton page (3.37 -> 3.18ms) and 2.7% on a full one -- the ~5% the profile predicted, not the 1.5-2x I guessed out loud. End to end it does NOT move bash's boot (3117 -> 3140ms, noise), because at 455 x 3.18ms the backend is now only ~1.45s of a 3.14s boot: the other ~54% is transport, Head construction and model bookkeeping on the client. Only the two hot sites changed. The four in the operand-format paths are not per-head and read better as is_code/is_data. Full gate: 1050 passed.
* Skeleton pages: stop rendering 227k rows to count them (3x boot)blasty2026-08-106-21/+129
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ListingView._grow streams the entire segment in the background for one reason: to learn how many rows it has, so the scrollbar and paging are right. It did that by rendering every row in full -- 227,500 rows of a 1.2MB bash, 911 backend calls, 9.3 seconds -- essentially none of which is ever looked at. generate_disasm_line is 22x the cost of the walk around it, so heads() gains text=False: a SKELETON page with the same rows at the same addresses with the same kinds and sizes, and no rendered text. Measured identical structurally (rows, addresses, kinds, sizes and cursor all match a real page) which is what makes one swappable for the other later. It also skips the digest (nothing to go stale) and lets the client skip the bulk opcode read, so a page costs ONE round trip instead of two. Client side is deliberately tiny, because the machinery already existed: a skeleton page is just a page whose text is stale. It is marked with a sentinel generation no _text_gen can equal, and the FIRST read of it goes through the same _ensure_text/_ensure_page path a rename uses -- which already refetches a page by address, verifies the structure still lines up and splices it in. Two staleness gates learn to fire for _skeleton as well as _renamed; that is the whole integration. bash boot: 911 calls / 9.26s -> 456 calls / 3.12s, 3.0x. The trade is that a page you actually display is fetched twice (3.3ms + 9.4ms vs 9.4ms), paid only for what is shown. _prime still loads real pages, so the viewport you land on is never a skeleton. The failure mode is BLANK ROWS, not an exception, and nothing in the suite scrolled far enough to see one: _prime renders the first ~1000 rows for real, so a test that pages down a few screens passes against a completely broken implementation. The new scenario reads deep rows through both the model and the render path, and asserts materialising changes neither the row count nor the walk. Verified by reverting the two gates: it fails with text=''. Full gate: 1050 passed.
* remote_tools: delete a shadowed duplicate hiding an lru_cache landmineblasty2026-08-091-74/+4
| | | | | | | | | | | | | | | | | | | | | | | | _idatui_head_row and _idatui_line_parts were each defined TWICE at module scope. Python keeps the last, so the first 74 lines were dead -- but the two copies of _idatui_line_parts were not equivalent: only the live one carries @lru_cache(maxsize=_IDATUI_LINE_CACHE). That makes the duplication worse than clutter. Anyone tidying it up by deleting the second copy would keep a byte-identical function and silently lose the most important cache in the backend: 10.4us -> 3.9us per listing row, ~2.7x on every row of every page. A comment now marks the spot and says which copy to keep. Found while profiling the digest; an AST duplicate-check reported the two as IDENTICAL because ast.get_source_segment on a FunctionDef excludes the decorator list. Compare decorators explicitly. Verified before deleting: the block defines only names redefined below, and nothing above it referenced them. Behaviour unchanged -- heads(500) still 9.3ms, digest still stable, and the expect/short-circuit contract still omits rows on a match and returns them on a miss. Full gate: 1042 passed.
* remote_tools: hoist the IDAPython imports to module scope (~3%)blasty2026-08-092-50/+121
| | | | | | | | | | | | | | | | | | | | | | | | | _idatui_head_row ran 'import ida_bytes/ida_lines/ida_name' on every LISTING ROW -- 25,500 sys.modules lookups per 500-row page. 50 such imports across the file, all pointless: this module is never imported by the client (codemode_client reads it as TEXT and installs it in the database process), so the no-IDA house rule that forces function-local imports elsewhere does not apply here. Measured honestly, it is worth about 3%: heads(500) best-of-40 goes 9.37ms -> 9.07ms, median 9.73 -> 9.65. A microbenchmark of the bare import predicted 3.2ms/page and was 10x optimistic -- the A/B is what counts, and it is a small win, not the big one. ida_hexrays deliberately stays function-local: it is licence-dependent, and hoisting it would break the whole remote library for someone without the decompiler rather than failing only when they decompile. Verified by AST that every ida_* reference still resolves. Adds experiments/profile_remote.py, which ships cProfile INTO the database process -- the only way to see the split between IDA's own calls and our python, which no client-side timer can show. Full gate: 1042 passed.
* experiments: count backend round-trips per user actionblasty2026-08-091-0/+164
| | | | | | | | | | | | | Answers 'are we batching or paying a round-trip per item' with numbers. Wraps CodeModeClient.invoke on a live pilot and reports calls + wall + ops for each action. The trap it encodes: ListingView._grow streams the whole segment on a worker thread, so it lands calls continuously regardless of what the user does. A naive census attributes that to whichever span is open and every action comes out at a near-identical '~1 call per 10ms of pause', which measures the pause and not the action. It now drains the grower first, reports it as its own line, and measures the rest against a quiet backend.
* PgUp/PgDn page the list overlays, not just the code viewsblasty2026-08-092-85/+200
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The help screen has always advertised 'PgDn / PgUp - page down / up', but only the four code views implemented it. In the palettes the keys did nothing at all: those screens focus a filter Input, so the OptionList's own pageup/pagedown bindings never fire -- every key goes to the Input, and an unhandled one is silently dropped. Adds OptionListNav, a mixin carrying the forwarding actions, and puts the six Input+OptionList overlays on it: symbols, search, strings, registers, load options, projects. They already held six BYTE-IDENTICAL copies of action_cursor_down/up, so this removes more than it adds. Paging delegates to the widget's own action_page_up/down instead of moving by a guessed N: those know the live viewport height, skip disabled options and clamp at both ends -- and it keeps the forwarded panes behaving exactly like the ones that page natively. Two panes deliberately stay off the mixin: * XrefsScreen focuses its list, so Textual already pages it. Now covered by a test so nobody 'fixes' it into double-stepping. * StructEditor binds ctrl+n to 'new type', so it cannot take NAV_BINDINGS; it gets page actions through its existing filter-focused guard instead. BINDINGS do not merge from a plain mixin (Textual only merges them from DOMNode subclasses), so every screen splats *NAV_BINDINGS explicitly -- the same trap SearchMixin documents. Tests gate on scrollable_content_region.height >= 1 first: paging is geometry, and before layout the page size is 0, so every check would pass against a no-op. Verified by removing the bindings again -- 3 checks fail with highlighted=0, which is the exact silent failure being fixed. Full gate: 1040 passed, 0 failed.
* README: drop the Docs sectionblasty2026-08-091-15/+3
| | | | | | | | | | | | | | | | | | docs/ is internal design and porting notes, not end-user documentation, so the README should not send readers there. Put it back when there are docs worth linking. Removed the section and the two inline pointers that fed the same files (-> docs/GRAPH_VIEW.md on Graph, -> docs/RPC.md on RPC), since the reason applies to those equally. No docs/ reference remains. Kept, deliberately, two links that are not end-user docs: * .agents/skills/idatui/SKILL.md -- promoted from a subsection of Docs to its own heading so removing the parent did not silently delete the only pointer to the skill we just shipped; * .fastfeedback/SPEED.md -- contributor-facing, sits inside Tests next to the commands it qualifies, and is accurate. Say the word if either should go too.
* README: cut the wall of text out of 'What's in it'blasty2026-08-091-67/+37
| | | | | | | | | | | | | | | | | | | | The feature coverage was right, the prose was not: eleven paragraphs that explained MECHANISM in an intro document. The keys table right above it already proved that scannable beats thorough here. Every feature is now name + key + at most two lines. Longest block went from ~70 words to 35; the section from 80 lines to 53. What got cut was rationale, not information, and none of it is lost -- each one is already the docstring of the module that implements it: why findings has to journal edits (journal.py), how search guesses text vs bytes (search.py), trail painting and unseen bytes as ?? (trace.py), why terminal graphics support is asked for rather than sniffed (kittygfx.py). The README now says what a feature IS and links onward. Also folded hex/strings/palettes/projects/splash into one 'Also' line (all already in the keys table), and gave the same treatment to the Run prose and the agent-skill paragraph.
* Ship the idatui agent skill in-repo at .agents/skills/blasty2026-08-091-1/+17
| | | | | | | | | | | | | | | | | | | | | | | | | Follows the Agent Skills standard (agentskills.io) rather than a contrib/ dir: harnesses discover .agents/skills/<name>/SKILL.md automatically from the repo root, and it is vendor-neutral (not .claude/ or .pi/ specific). Not a verbatim copy of the local one -- it was ~85% generic and the rest had to go before redistributing: * the backend is described as a normal pyproject dependency, with the editable-git-checkout setup demoted to a conditional note (that is one maintainer's layout, but the 'upstream pull silently swaps the backend' warning is worth keeping for anyone who does it); * personal agent tooling (bgrun/waitfor wrappers) replaced by the underlying lesson -- a backgrounded subshell holds the inherited stdout, so redirect all three fds -- which is true without those scripts; * maintainer commit authorship dropped; * absolute ~/dev paths made relative to the repo root; ~/ida-venv kept but marked as this project's assumption, matching the README. Also fixes the README's stale full-run count (788/~2m -> 1031/~50s), lists the docs that were missing from the index, and warns that a user-level skill of the same name shadows this one -- otherwise a personal copy silently wins and the in-repo copy rots.
* SPEED: replace the historical backend table with a real 0.3.1 vs 0.3.2 A/Bblasty2026-08-091-0/+103
| | | | | | | | | | | | | | | | | | | | | | | | The worker-vs-Code-Mode table was measured before 0.3.2 and with both workarounds active, so it answered a question nobody asks any more. Replaced with three configurations measured on the same box, rolling both checkouts back and forward: A old client WITH workarounds on 0.3.1 -- what shipped B current client on 0.3.1 -- what the workarounds were for C current client on 0.3.2 -- now Headline: the real-world gain is ~1.4x geomean, NOT the 6.9x the empty round trip advertises, and the doc says so in those words -- because the tempting number to quote is the wrong one. The A->C vs B->C gap is the actual story: stock 0.3.1 was 5.4x slower, so the workarounds had already recovered nearly everything and upstream mostly bought us the right to delete them. Also records the three cost classes (payload- / round-trip- / IDA-dominated) so the next person optimising here knows which lever moves which op, and the ~10% run-to-run spread so a sub-1.2x 'regression' doesn't start a hunt. Old worker table kept below, labelled historical. experiments/bench_ops.py is the harness, with the copy-to-/tmp-before-checkout trick documented in it.
* Docs: re-check ALL nine upstream findings against 0.3.2blasty2026-08-091-14/+49
| | | | | | | | | | | | | | | | | | | | | Items 1-2 were marked fixed; 3-9 were left 'not re-verified', which was a gap rather than a finding. Checked all seven. Item 3 is also FIXED, and by a lot: a same-box A/B (roll the editable checkout back to 4195f21, measure, roll forward) puts execute_python('result = 1') at 2.055ms on 0.3.1 and 0.294ms on 0.3.2 -- 7.0x, now about the cost of a bare HTTP GET. The 0.3.1 column reproduces the originally documented 2.025ms almost exactly, which is what makes the 0.3.2 column believable. So the execute_sync marshalling that was ~93% of the floor is gone, and the 'never make a call per row' rule is much weaker. Items 4-9 are open by construction: client/registry/resolver/server/ database/worker .py are byte-identical between 0.3.1 and 0.3.2, and all six items live in those files. Recorded that as the cheap re-check recipe for the next upstream pull. SPEED.md taught the obsolete 2ms floor as a design rule; corrected, with a warning not to go rewrite working batched code to be chatty.
* Docs: correct the stale --fast check counts (302/257 -> 380)blasty2026-08-092-2/+2
|
* Docs: mark the two upstream perf findings fixed in 0.3.2blasty2026-08-091-3/+24
| | | | | | | | CODEMODE_UPSTREAM items 1 (timeout_trace) and 2 (to_jsonable) both landed upstream; note it at the top, on each item and in the priority table, and flag that items 3-9 are not re-verified against 0.3.2. SPEED.md described the settrace workaround and its IDATUI_CODEMODE_TRACE knob as current -- both are gone, and its backend table predates 0.3.2.
* Drop the settrace workaround: ida-codemode 0.3.2 deleted the hookblasty2026-08-092-41/+125
| | | | | | | | | | | | | | | | | | | | _script() ran every snippet with sys.settrace(None) to detach Code Mode's per-line trace hook, which returned itself and so enabled line tracing in every frame it saw -- 52x on ida_bytes.get_flags, and the whole residual gap against the old private worker. 0.3.2 removes that hook entirely (zero settrace refs); the deadline is now a C-level thread interrupt, runtime._interrupt_thread. Re-measured on a 200-row listing page of main in targets/bash: the workaround buys 0.99x. So it goes, and the caveat goes with it -- a pure-Python loop inside a snippet is once again bounded by its deadline. _PACK_EPILOGUE measures 0.97x for the same reason (dumps_json got a C fast path) but is kept for encoder determinism, with its stale 114x claim corrected. experiments/bench_pack_trace.py is the harness for both numbers. Full gate green: 1031 passed, 0 failed.
* Graph: never hand triskel a block its root cannot reachblasty2026-08-093-58/+125
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | "EMPTY BL" in the status bar is triskel's own bracket-list assertion from its SESE pass, and it turned out to be the mild version of the problem. Triskel's graph root is whichever node was created FIRST, and every analysis walks out from it. Anything unreachable from that node is undefined behaviour. We were: - creating nodes in id order, so the root was the lowest-numbered block rather than the entry, and - splitting only WEAKLY connected components, which says nothing about reachability. A 7-block CFG whose entry has no successors -- IDA hands those out for thunks and for dead code it could not resolve -- SEGFAULTS the interpreter. That is unsurvivable: it takes the session down and there is no exception to fall back from. Now the entry is created first, orphan blocks are attached to it with phantom edges that steer placement but are never drawn (one edge usually adopts a whole orphan subgraph, attached at a node no other orphan reaches), and reachability is asserted in python BEFORE crossing into C++. This replaces the component splitting entirely: one layout instead of N stacked side by side, and triskel gets to place the orphans. The reproducer is now a test (t_unreachable_entry). Remaining fallbacks on the ls corpus are 8/1200 layouts, all the upstream box-overlap defect, all but one on 300-500 block functions.
* Graph: fix the triskel fallbacks, and say why when it still falls backblasty2026-08-095-32/+181
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "engine triskel -> native+triskel-failed" was unreadable and, worse, gave no reason: the cause went to a logger a TUI user never sees. Dumped all 400 functions of bin/ls (echo's 128 were not enough) and swept them at three zoom levels: 6 of 1200 layouts fell back, all of them my own _verify tripping over a detour that could not be placed. - the detour jumped to the nearest side of the FIRST box in the way, which in a dense layout is usually inside the next box along. It now collects every box the run passes and picks the nearest genuinely free line. - it skipped the first and last segments because they carry the port and the arrowhead. But that is exactly where the failures were: triskel is happy to park a block directly above its successor and drive the final approach straight through it. Those segments may now move ALONG their own box's border, which is free almost every time. - repairs are swept to a fixed point: moving one segment stretches its neighbours, which can push those into a box. 0/1200 fallbacks after that. Then the bigger corpus turned up a second, genuinely upstream defect: superimposing SESE regions can leave two blocks a couple of columns into each other (2 of ls's 400 functions, in float space, before rounding). Cosmetic in a PNG; here the boxes are made of text, so one block's disassembly overwrites another's. _verify now checks it and falls back, which is the right trade. Reporting, so this is never mute again: - stats["engine_error"] carries the reason, the status line shows it, and the label is "native (triskel failed)". - the corpus test asserts fallbacks are rare AND explained, rather than asserting they never happen. Also lowered AUTO_TRISKEL_MAX_BLOCKS 250 -> 180. Layout runs on every zoom keypress and triskel knees hard past ~175 blocks (174: 66ms, 233: 489ms, 329: 555ms). The old cap allowed a 489ms stall. The corpus timing check now measures only sizes `auto` can actually reach, plus a 5s ceiling so nothing blows up quadratically when forced.
* Graph: name the interpreter when pytriskel is missingblasty2026-08-092-2/+17
| | | | | | | | | ./ida-tui runs $IDATUI_PYTHON (default ~/ida-venv), which is not the repo .venv the tests use -- so the graph view can report pytriskel as missing while tests/test_graph.py is exercising both engines. "not installed" on its own sends you to check the wrong python; the status now names sys.executable, and docs/GRAPH_VIEW.md says to install into both.
* graph_triskel: name the detour target, drop the unused swapblasty2026-08-091-8/+8
|
* Graph: a second layout engine, triskel's SESE decompositionblasty2026-08-0911-57/+1054
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `e` in graph mode cycles auto -> native -> triskel, and `auto` prefers triskel where it is installed and the function is at most 250 blocks. Why: our layered engine draws wide-and-short pictures with a lot of crossings on anything branchy. Triskel splits the CFG into Single-Entry Single-Exit regions first and lays each out on its own, which on the 128-function corpus means fewer crossings on 12 functions, equal on 9, worse on 3 -- and the wins are the hairballs (sub_5CA0 41 -> 6, sub_2C90 32 -> 7, sub_2C00 12 -> 0). It also routes loop edges around the side of the graph the way IDA does, which was a known gap here. It is not free: ~2x slower at 87 blocks, 10x at 424, hence the cap. The library needed a fork (~/dev/triskel, branch idatui) before it could be used from Python at all -- its get_waypoints() threw on every published version, an empty graph segfaulted the interpreter, and its spacing constants were pixels baked in at compile time. Making those settable is what makes this integration cheap: we hand it CELLS, so its output is integral and two edge lanes can never round onto the same row. The feared quantisation problem measured out backwards -- cells claimed by more than one edge: native 131, triskel 35. Not trusted with degenerate input, all handled before the call: self-loops and disconnected components make it throw, and one corpus edge comes back routed through a block, which we detour and re-verify. A triskel failure is never fatal; it falls back to native. Two things the second engine flushed out of the existing code: - the canvas was sized from boxes alone, which is exact only because native's dummy nodes reserve the space. Triskel routes outside that bounding box and the edges were being clipped. - arrowhead placement read e.back, conflating "this is a loop edge" (style) with "this polyline runs against control flow" (geometry). Now Edge.flipped, which is also a latent fix for residual-cycle edges whose succ/pred were being reported backwards. tests/test_graph.py runs its whole suite once per available engine (943 checks); new graph_engine scenario covers the live toggle.
* splash: scale the logo to the pane instead of dropping itblasty2026-08-073-10/+105
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reported as "the splash logo stopped rendering". It had not stopped: the splash asks for the artwork's NATURAL size and shows nothing when that does not fit, and the artwork needs 31 rows plus 10 of box chrome. A pane in a split zellij window is 31 rows — one row short of the 41 it wanted — so the logo silently disappeared. Traced with $IDATUI_KITTY_LOG in the real session: compose: supported=True app.size=Size(width=159, height=31) cells=60x23 fits=False The terminal scales an image into whatever cell box it is placed in (`c=`/`r=` on the placement), so there was never a reason for all-or-nothing. `logo_cells(max_rows)` now fits the art to the room left after the box's furniture, and the same number reserves the cells and sizes the placement, so a resize needs no relayout. In that same 31-row pane it now draws 55x21 instead of nothing. Two things fixed on the way: * The chrome constant was one row optimistic (`rows + 9` where the box measures 10: border 2, padding 2, art margin 1, title 1, note 1+1, help 1+1). At exactly the old threshold the help line was clipped off the bottom. * `_fits` conflated "is the terminal big enough" with "is the artwork the right size", which is what made the image path inherit the block art's all-or-nothing behaviour. The block art genuinely cannot scale (it is half-block cells, 26 rows) and still falls back to the text splash; the image no longer does. `splash_scaling` pins it at 31, 30 and 44 rows: the logo is drawn, it is scaled to the room, the box is never clipped, and a big pane still gets the natural size. 905 passed, 0 failed.
* Centre modals with a rule about modals, not a list of themblasty2026-08-073-10/+63
| | | | | | | | | | | | | | | | | | | | SearchPalette opened pinned to the top of the screen: the CSS named the screens that centre (`SymbolPalette, StringsPalette, ProjectPalette, …`) and a new dialog is not on a list nobody remembers to edit. The comment sitting above that rule — "every #pal-box palette centres, not just the symbol one" — was the *first* time this happened. `ModalScreen { align: center middle; }` matches subclasses, so every dialog inherits it and the next one is centred for free; the eight per-screen rules that only repeated it are gone. Textual's own Ctrl+P CommandPalette is a ModalScreen too and wants its stock top alignment, so it opts out in one visible line rather than by omission. The `modal_centering` scenario checks both halves: that centring is expressed as a rule, and that it actually reaches a dialog's laid-out region (above/below and left/right within a cell). 894 passed, 0 failed.
* Ctrl+F: search the whole database, by text or by bytesblasty2026-08-0710-4/+728
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `/` only ever searched the lines of the view you were in. This adds the search you actually need on a binary: over the entire database, either through the rendered disassembly or through the image. * **text** matches the line as displayed, whitespace-normalised, so `call cs:` finds `call cs:getenv_ptr` (IDA's column padding is not something anyone types). Smartcase; `regex` available over RPC. * **bytes** is IDA's own `find_bytes`, so the pattern language people already know works unchanged: hex pairs, `?` wildcards for a whole byte or one nibble (`48 8? ?? 24`), quoted literals (`"Hello", 0`). Commas, no separators (`488B05C3`) and ragged spacing all normalise. **Which mode you meant is guessed, and the guess is biased on purpose.** `dead`, `add`, `cafe` and `ff` are valid hex AND ordinary things to search for, so a bare hex-looking word stays TEXT; nobody types `48 8b ?? c3` meaning prose. `hex:`/`text:` prefixes and F2 override it. The subtle case is a *typo* in a byte pattern. `48 zz c3` first fell through to a text search and reported "no match" — indistinguishable from "those bytes are not in this binary", which is the most misleading answer a search can give. Now any query whose tokens are all byte-sized is treated as bytes, and a bad token is refused BY NAME. IDA does the same thing quietly (find_bytes answers a malformed pattern with zero hits and no error), so the validation lives in Program.search, not just in the UI. Enter searches, then Enter opens the highlighted hit; the title says which it will do, because a database-wide scan is far too slow to run on every keystroke like the other palettes. Navigation goes to the item head — a byte match can start mid-instruction — and the status names the exact address. Also: the `find` RPC verb and `drive find`, which is the one an agent wants (`drive find '48 8b ?? c3'`). idatui/search.py holds the classification and is pure, so the whole question of "what did they mean" is tested offline: tests/test_search.py, 35 checks, 0.1s. Pilot scenario db_search covers the UI end to end. Full suite: 890 passed, 0 failed, 51.3s.
* Export findings as markdown (Ctrl+E), and the journal that makes it trueblasty2026-08-0712-6/+1077
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The output of an RE session is what you worked out, and it was locked in a .i64 that only IDA can read. Ctrl+E (or `drive export`, or the `export` RPC verb) writes it out: your comments grouped by function with the line each annotates, the names and prototypes you set, the types you declared. **The hard part was provenance, and it needed a mechanism, not a filter.** A database does not record WHO wrote a comment or a name. IDA's analyzer sets `; switch 73 cases` and `; s1` with the same `set_cmt` a person uses, and the ELF loader sets `elf_gnu_hash_nbuckets` and `File class: 64-bit` the same way. Four probes, all negative: the FF_COMM flag is identical, `get_cmt` returns them all, `generate_disasm_line` tags every one of them COLOR_REGCMT (not COLOR_AUTOCMT), and they survive with auto-comments switched off. A first cut filtered by shape and produced a report whose first screen was ELF header trivia and `; jumptable ... case 99`. So idatui journals its own edits (idatui/journal.py) into a netnode in the database: it rides along in the .i64, it is still there next session, and the report is then exactly what was done here -- 2 findings out of a database carrying 693 other annotations. Recorded at the choke points in edit_ctl (rename, name-address, comment, retype) and in the struct editor; flushed on save, on export and on quit, so no edit pays a round trip. Without a journal (a database worked on in the IDA GUI, or predating this) the report falls back to filtering by shape -- dummy names, imports, loader segments, the analyzer's stereotyped switch/jumptable strings -- and says so in the document rather than claiming authorship it cannot prove. idatui/findings.py splits gather (needs IDA) from render (does not), so the formatting, grouping, sorting, escaping and the empty cases are tested offline: tests/test_findings.py, 32 checks, no worker, 0.1s. The pilot scenario covers the round trip that matters -- edit through the UI, export, find it in the file, and reload the journal from the .i64. Full suite: 842 passed, 0 failed, 51.2s.
* tests: gate on real signals, not sleeps (117s -> 49s)blasty2026-08-076-60/+261
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The suite spent its time in two kinds of guess. **Flat pauses.** ~140 `pause(d)` calls were 20.4s of the pilot's 62s, and `test_trace_ui` was 13.5s of `pilot.pause(1.0)` out of 19.6s. `Ctx.pause` is now `settle` (`d` is the upper bound, not the cost) and the other suites' sleeps became gates on the thing the check is about. `Ctx.sleep` stays for what a timer really drives. **Textual's keypress path.** `Pilot.press` calls `wait_for_idle` twice per key, which sleeps in 20ms granules until process time stops advancing -- 84ms per keypress here, 23s of the pilot's 43s. `_fixtures.fast_keys()` replaces it with the gate the suites already use: send the keys, then settle. Deleting the heuristic *without* that broke nine checks, so it was doing a job, badly. Four checks turned out to be riding on those sleeps: they read geometry or a repaint (`si.region`, `gv._minimap_rect()`, glyphs off `gv.render_line`, a repaint trace), and a settled app has not necessarily been laid out or painted. They now wait for the frame. The debounced function filter (`set_timer(0.08)`) likewise waits for its effect. Also fixed two waits on signals that never arrive: the comment wait in `rename` carried a `dec.loaded_ea == app._cur.ea` conjunct that cost 9s of timeout and then let the check pass vacuously, and `listing_view` -- the one entry under "Known-flaky" -- waited on `lst.total`, which is true before a single row exists. `--profile` reports, per scenario, seconds settling / waiting / pressing, and names any wait that expired with its line number. It is how the above was found and how the next 20s should be. Verified: 4 full `tests/run.py` runs, 800 passed each, 49.0-49.2s (was 117.4s); 4 consecutive pilot runs, 313 passed each, 21.2s (was 63.7s).
* Struct editor: '/' fuzzy-filters the struct listblasty2026-08-073-10/+221
|
* Modal chrome: thin round borders, titles in the borderblasty2026-08-073-56/+223
|
* Syntax-highlight the struct editor's C definitionsblasty2026-08-074-14/+143
|
* demo: tighten the pacing (43s of pauses -> 19s)blasty2026-08-071-38/+38
| | | | | | | | | | Manual navigation is snappy; the demo was not, because I authored the pauses for someone reading every status line. Halved every beat with a 0.25s floor so the scenes stay distinguishable, and dropped the prompt typing from 45ms to 18ms per character -- still legibly typing, no longer something you wait on. --speed still scales it either way: 0.5 for a fast cut, 2.0 to slow it back down for a narrated take.
* demo: do not re-navigate to main when the app already landed thereblasty2026-08-071-3/+11
| | | | | | | The app auto-lands on main once the function index is in, so scene_open typed a goto to a place it was already standing. Every keystroke a driver injects costs a round trip, so that was pure dead time at the very start of the tour -- the worst place to spend it, since it is the part a recording opens on.
* demo: --here, render the TUI in the terminal that ran the demoblasty2026-08-071-13/+89
| | | | | | | | | | | | | | | | | | | | | | | | | --spawn needs tmux or zellij and puts the app in a second pane, which is wrong when the capture is one terminal. --here launches the TUI through the ./ida-tui launcher with stdio inherited, so it draws on the terminal you ran the demo from, then drives it over the socket exactly as before and quits it at the end. Going through the launcher rather than re-deriving the interpreter keeps the $IDATUI_PYTHON rules in one place. Two things this mode has to get right: - Narration would be drawn over the app, so it is collected and replayed after the alternate screen is gone. The operator still sees the full scene list, just at the end instead of during. - The socket is waited for with a deadline that also watches the child, so a TUI that dies before it ever listens fails in a second instead of hanging until the timeout. Teardown reaps the process (wait, then terminate, then kill) and removes the scratch copy on every path. --here and --spawn are rejected together. Verified: argument handling, the mutual exclusion, and that the pure suites are unaffected. The end-to-end --here run is NOT verified here -- it needs a real terminal, which this environment does not have.
* tools/demo.py: a scripted feature tour, and the RPC bug it foundblasty2026-08-073-1/+339
| | | | | | | | | | | | | | | | | | | | | | | | | | Ten scenes over the RPC layer: goto, listing scroll, pseudocode, the literal format ring, follow/back, the graph (zoom, minimap, edge walking), split view, xrefs, a rename and a comment, the strings/symbols/structs/hex browsers, and search. It drives the semantic verbs, so prompts are typed character by character and a recording shows the app being used rather than poked. --spawn opens its own pane on a COPY of the target (binary plus .i64, so the analysis is not re-paid) and tears it down afterwards; --sock drives a session you set up yourself, for control over pane size and zoom. Edits are reverted at the end, including a re-navigate so the last frame does not still show the demo's rename. --speed scales every pause, --only runs a subset, --list prints the scenes. Writing it found a real bug in the RPC hex verb: it waited for `app.is_hex` to become true, but backslash TOGGLES the hex view, so the call that leaves hex could never satisfy its predicate and always timed out after 20s. Any driver could open the hex view and never close it. It now waits for the mode to flip, the way toggle_view already did. Verified by running every scene against a live database and asserting the UI actually moved -- landed on main, decomp reachable, graph reports 280 blocks, split toggles off, no modal left open, the rename reaches the database and the revert removes it. 15/15. Full suite still 788 passed, 0 failed.
* Install ida-codemode from PyPI, not a sibling cloneblasty2026-08-073-27/+20
| | | | | | | | | | | | | | | | | | | | | | | | The README told people to "git clone ... ../ida-codemode" and pyproject carried a matching [tool.uv.sources] path override. That is not a convention, it is Duncan's development layout, inherited from the port patch and never questioned: it writes into the parent of the user's checkout, breaks if the directory is named anything else, and pins everyone to whatever happens to be in that working tree. ida-codemode is published on PyPI (0.3.1, the same version the clone was sitting at), so the dependency is now an ordinary versioned requirement and "uv sync" is the whole install. Verified end to end: uv lock resolves, uv sync --frozen installs from the registry, and ida_codemode.client.DatabaseHandle imports from the resulting venv. The path override is still the right answer when hacking on both at once, so that stays in the README as one "uv add --editable" line rather than as the default everyone pays for. Also fixed the plugin command it now documents: upstream's own README passes --prerelease=allow, and without it uvx refuses to resolve, because ida-codemode depends on ida-domain>=0.5.1.dev2. Checked that --install-plugin is a real flag and that the command runs, instead of copying it over on faith.
* docs: drop two stale files, correct the two the README links, cut sharp edgesblasty2026-08-075-542/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | README: "Known sharp edges" removed. docs/GRAPH_VIEW.md pointed the graph's one backend call at server/patch_server.py, a file the port deleted. It is idatui/codemode_client.py now. This matters because GRAPH_VIEW.md is one of only two docs the README links -- a wrong path in a linked doc is worse than no doc. docs/RPC.md was missing five verbs rpc.py serves: ping/methods/quit (now a Session table) and trace/binaries/switch (with the semantic verbs). The trace verb drives a whole feature the README advertises, so its absence was the real gap. Every verb in rpc.py's METHODS table is now documented, checked mechanically -- the remainder of what looked missing turned out to be values of the "move" verb's dir parameter, not verbs. Deleted: - docs/CODEMODE_PORT.md. Written mid-port, and its Result section claims the port covered "the complete current TUI feature set". It did not: the graph view was dead, the operand-format keys were dead, pseudocode comments never appeared, and bulk rename was broken. Keeping a doc that states the opposite of what we found is worse than having none, and docs/CODEMODE_UPSTREAM.md carries the accurate account of the API limits. - docs/TUI_DRIVING_BLUEPRINT.md. A design sketch for generalizing the drive-a-TUI-over-a-socket pattern to other programs, with gdb as the main example -- which now exists for real in ~/dev/gdb-driver. Superseded by the thing it proposed. Nothing linked either file. Both remain in git history.
generated by cgit v1.3.1-sl0p (git 2.54.0) at 2026-08-28 09:48:09 +0000 — sl0p fork