| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
multiline comments)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Both exist because an in-process Textual pilot passes on things a real
terminal does not do.
type_bench.py times the semantic verbs at several delay_ms values through
a real pty -- tests/_fixtures.fast_keys patches the per-key idle wait
away, which is exactly the cost being measured, so the pilot is blind to
it. This is where the ~80ms/char figure behind `press()` came from.
click_pty.py injects SGR mouse bytes on stdin like a terminal emulator
and reads the cursor back over the RPC. `pilot.click()` posts
MouseDown/MouseUp/Click straight at a widget and bypasses Textual's own
click synthesis, so it can pass while a real click does nothing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`App._press_keys` waits on Textual's `wait_for_idle` twice per key -- a
CPU-load heuristic that sleeps in 20ms granules until process time stops
advancing. Measured through the RPC server on a real pty that is ~80ms
per character, so `delay_ms=35` typed at ~115ms/char (a third of the pace
it named) and `delay_ms=0` -- what `idatui.drive` asks for, and what an
agent wants -- was no faster than a slow human.
`press()` swaps the heuristic for a bare yield while our keys go in, the
same trade tests/_fixtures.fast_keys already makes. What replaces it is
not "nothing": every verb ends in settle() (pump drained, threaded
workers finished), which is the real gate and strictly stronger than "the
CPU looks idle", and `wait:<ms>` tokens still sleep for real so a delay
that was asked for still renders frame by frame. It patches around the
call rather than at import, so nothing else -- the pilot suite in
particular -- changes behaviour if this module is imported there.
With the floor gone, the default delay is the whole cost of a keystroke,
which makes 35 dishonest as well as slow: TYPE_DELAY_MS is 20 (~50
chars/second, still visibly typed). docs/RPC.md says what delay_ms now
means, since "0 is instant" only became true here.
Also moves the last two `client.invoke("name", ...)` call sites in this
file to `client.call(remote_ops.name, ...)`, which is what the rest of
the tree uses since the ida-nexus port: the operation is a reference, not
a string nobody checks.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The slosh backend worked out "where do I live" from `panes[].focused`.
Exactly one pane in a session is focused, it is always in the tab
currently on screen, and it belongs to whoever is looking -- so a TUI
spawned "beside the agent" landed in whatever tab the human had wandered
into, and in a session running several agents it was reliably a
stranger's. `apply-layout` jumping the view to the tab it built made it
self-inflicted too: one spawn moved the focus the next spawn would read.
Identity now comes from four sources, in order, none of which is "whoever
has focus": $IDATUI_SLOSH_PANE (an operator saying so), $SLOSH_PANE
(slosh telling the pane's program which pane it is), our own /proc
ancestry matched against panes[].pid (the answer for an agent's shell-out,
which has neither a scrubbed environment nor a controlling tty), and a
purpose named by $IDATUI_ORIGIN_PURPOSE / $SLOSH_PANE_PURPOSE. If none of
them can answer, the pane stays in the tab the layout built it in --
visible, findable by purpose, and nobody else's tab -- and says so, since
the fix is one variable in the caller's environment.
The id-addressed verbs are probed, not assumed, and the probe is a field
rather than a verb: slosh ignores arguments it does not know, so `raw`
with an `id` on an older build answers ok and writes the bytes into
whatever pane has focus -- silently, into somebody's shell. panes[].pid
arrived with those verbs, so its presence is the capability. Where they
exist: `raw id=` writes without touching focus, `capture id=` reads a pane
in any tab without hopping the view (the old snapshot-and-slice route is
kept as the fallback, and is rude precisely because it moves what other
people are looking at), `apply-layout focus=false` builds the tab without
jumping to it, and `move-pane beside=` names our pane rather than letting
slosh pick that tab's focused one.
Detached is now the default and `--focus` is the opt-in. The other way
round, every spawn ended by pulling the session's focus -- and so the
viewed tab -- to a pane nobody asked to look at: fine when a human typed
the command and is waiting for it, wrong for an agent opening a binary
while somebody works two tabs away. `--detached` still parses, for callers
that say it.
tests/test_pane_origin.py (22 checks, pure stdlib, no IDA and no slosh:
the decisions are made about a `panes` reply, so the reply is the input)
pins the precedence order, the focus-is-not-identity rule, the ancestry
walk and that an unanswerable session leaves the pane where it was built.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`pane spawn` knew tmux and zellij, so under slosh every entry point
answered "error: not inside tmux or zellij" — the agent skill described a
workflow that could not run. slosh is now the first thing autodetected
($SLOSH, then $ZELLIJ, then $TMUX), and a slosh pane id is a bare integer,
so a registry row written before this commit still routes its own
stop/capture/keys.
Everything goes through the control socket, always the JSON form
($SLOSH_BIN -s $SLOSH_SESSION cmd): the bare-verb alias answers unwrapped,
and a parser written for one shape silently finds nothing in the other.
$SLOSH_BIN rather than `slosh` because a session may have been started
from a build tree that is not on $PATH.
Spawning is the one place slosh differs in kind. Its `split` verb only
ever makes a shell pane, and typing a command into somebody's shell is
the thing not to do — a pane that was *given* a command is the one with
alive/exit_code to poll and `rerun` to repeat. So spawn applies a one-tab
layout whose pane carries command= and a declared purpose=idatui:<hex>,
finds it by that purpose (declared purposes are locked, so the TUI cannot
relabel itself out from under us, and titles change anyway), then
move-panes it into the tab we were called from: it lands beside the
caller with its pty and scrollback intact, and the layout's now-empty tab
removes itself. --detached restores focus afterwards, since slosh focuses
what it creates and moves.
The rest follows the zellij precedent. A dead command pane stays on
screen as a husk, which _pane_alive must read as dead or a launcher that
died on startup would burn the whole 300s readiness timeout instead of
reporting "pane exited during startup". `capture` cuts the pane's
content_* rect out of `snapshot` (which is the whole composited session)
and hops tabs if the pane is parked in another one, because only the
active tab is composed at all. `keys` keeps taking tmux-flavoured names
and turns them into the bytes a terminal sends: it focuses the target,
writes with `raw`, and restores focus — `send` would be decoded as
session input first, where C-a is slosh's own leader and never reaches
the TUI. --size stays tmux-only and now says so for whichever mux is in
play.
The two RPC integration tests gated on $TMUX/$ZELLIJ, so they skipped
silently under slosh; they accept $SLOSH now.
Verified against the live session, not just in unit checks: a probe pane
spawns, reports alive, captures its own output, receives real keystrokes
and dies on close; `pane spawn --open targets/echo` reaches ready:true
with 128 functions, `drive where` answers, `pane keys Tab` flips listing
to pseudocode on screen, and `pane stop` leaves an empty registry and no
husk. Env inheritance is unchanged from zellij (a pane gets the session
server's environment, not the caller's) — the probe printed PROBE_VAR=[].
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`ida-tui /bin/ls` asked IDA to create /bin/ls.i64, which a normal user
cannot write, and what came back was the backend's account of it:
idalib worker launcher 2425195 exited with status 1
[ida-nexus] Failed to open database /usr/bin/ls
after a full analysis wait, naming neither the file it could not create
nor the reason. Nothing in the message points at permissions, and there
was no way forward from it.
idatui/idbpath.py answers the question before anything is opened: where
would the database go (mirroring ida_nexus's own derivation), can it go
there (os.access only — the pre-flight check must not create files), and
where instead. The proposal is stable per binary
(~/.local/share/idatui/db/<name>-<sha1(realpath)[:8]>.i64) so reopening
the same binary resumes the same names and comments, and hashed so two
busyboxes from different directories cannot collide. DATA_HOME rather
than CACHE_HOME on purpose: an .i64 holds the whole session, and cache
directories are something cleaners may empty.
The answer becomes NexusClient(output_database=...) — the option project
mode already uses to keep everything in its sidecar. `--idb PATH` (also
forwarded by `pane spawn`) says where without being asked. An RPC-driven
TUI is never asked: `pane spawn` polls for ready and would sit out its
whole timeout in front of a modal nobody can answer, so it relocates and
says so in the status line. A failed open re-offers relocation after a
real create probe, since os.access does not know about ACLs, a lying
network mount or a full disk.
tests/test_idbpath.py (pure, 32 checks) pins the path arithmetic, the two
properties of the proposal and that neither check touches the user's
directory; tests/test_idb_relocate_ui.py (textual, no worker, 24 checks)
drives the dialog, Esc, an edited path, the untouched common case, --idb,
the RPC path, and that output_database actually reaches the client.
Verified end to end against a real /bin/ls: 400 functions, and a rename
saved in one session comes back in the next.
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
https://github.com/blasty/ida-tui/pull/2 — pseudocode address anchors are
now split off only as the trailing /*0x…*/ our decompile tool appends, so a
marker-shaped string literal or source comment survives display and cursor
columns. The listing span walker asks the kernel (tag_skipcode) how long
each colour code is instead of assuming two bytes, so semantic group tags
with kind/TID headers stay invisible on kernels that emit them. Bumps
ida-nexus to 0.13.0.
Verified locally on IDA 9.4 + ida-nexus 0.13.0: full suite 974 passed
(955 on main + 19 new pure checks). Old vs new span walker give identical
output over 60k bash heads (no 0x36 tags on this kernel); the new walker
costs 19.8 vs 12.3 us/line, next to 7.6 us for generate_disasm_line.
|
| | | |
|
| | | |
|
| |/
|
|
|
|
|
| |
Plain-language intro, numbered getting-started steps, readable keys and
feature tables, advanced usage folded into a details block. The lease /
event-stream / remote-op details now live under 'How ida-tui talks to IDA'
in CONTRIBUTING.md so nothing is lost.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
PR #1's live refresh had client-level coverage only (the listener thread
and debounce, in test_nexus_client); the app half -- _refresh_idb_events
invalidating, reindexing, and reloading the active surface -- was untested.
The scenario renames straight through the client so no UI cache hears
about it (what another client's edit looks like from this process), hands
the app a synthetic batch, and requires: the status says why the view is
about to move, the listing rebuilds around the same cursor and viewport
address, the function index carries the external name, pseudocode
decompiled afterwards shows it, and a second batch landing while decomp is
active reloads it in place. That second batch also reverts the rename, so
the scenario is idempotent.
|
| | |
|
| |
|
|
| |
No behavior. Listed in .git-blame-ignore-revs (next commit).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
PR #1 arrived black-formatted, which forced the question. ruff's formatter
is black's output, it is one static binary rather than a dependency chain,
and it is fast enough that a hook is free. ruff.toml pins the version
(a formatter whose output moves between releases turns 'formatted' into
'formatted by whoever committed last') and selects only import sorting on
the lint side -- a hook that refused a commit over an unused variable would
be a different feature.
.githooks/pre-commit (install: git config core.hooksPath .githooks) formats
exactly what is staged, refuses partly-staged files rather than quietly
widening a commit built with git add -p, and prefers the pinned .venv ruff.
Also drops the blanket '# ruff: noqa' PR #1 left in remote_ops.py -- it
would have exempted that file from the only lint rule we enforce.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two scenario monkeypatch sites still hooked client.invoke -- the rename to
call() updated the call sites but not the patches, so reprime_is_free and
split_view crashed instead of counting. call() takes the remote_ops
declaration itself now, so both count by its __name__.
remote_ops imported RemoteModule inside _bindings(), which made
test_nexus_client (NEEDS_IDA = False) unrunnable under a stdlib-only
python3 -- the house rule tests/run.py --fast depends on. The import moves
to the same eagerly-if-present, bound-to-None-so-patchable contract
nexus_client uses, and the test injects FakeRemoteModule/FakeRemoteError
exactly like its other fakes: real names when the library is installed,
strict fakes when it is not.
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
https://github.com/blasty/ida-tui/pull/1 — from the ida-nexus (né
ida-codemode) maintainer: kitty-graphics fallback for platforms without
termios, Ctrl+R view refresh, typed remote operations through ida_nexus
RemoteModule, live auto-refresh from other clients' IDB events (upstream
item 7), discard-without-save (item 6), and the ida-nexus 0.7.0 rename.
tests/test_kittygfx.py is the union of both sides' files: our escape-
construction checks plus the PR's cross-platform fallback checks.
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A kitty placement is identified by (image id, placement id); an a=p with no
p= key is anonymous and every one stacks another copy. The splash re-anchors
on every progress note (~5/s), so a 60s load ended with ~300 placements of an
RGBA logo alpha-compositing over each other -- soft edges creeping to solid,
and the terminal re-rendering all of them per frame.
Same pair every time (LOGO_PLACEMENT) = the terminal REPLACES the placement,
so re-anchoring is free and atomic. That is also why on_resize no longer
clear()s first (that showed a hole for a frame), and why a shrunk-to-nothing
region now drops the placement instead of leaving a stale one anchored.
tests/test_kittygfx.py pins the escapes (pure, stdlib);
experiments/splash_place_count.py counts what a real splash sends.
|
| |
|
|
|
|
|
|
|
| |
The install instructions pointed at ~/dev/triskel, a path only I have, so a
reader could not act on them. Say plainly that the patched pytriskel is not
released yet and that native is the supported default.
Also drops the .fastfeedback/SPEED.md references and refreshes a stale
branch name and commit hash in bench_ops.py.
|
| |
|
|
|
|
|
|
|
| |
TRISKEL_EVAL.md and graph_compare.py were intermediary working docs from
choosing the SESE engine. Triskel now ships alongside the native layout
engine, so the evaluation no longer needs to be in the tree; GRAPH_VIEW.md
keeps the part that describes shipping behaviour.
Also drops the local-only ~/dev/triskel/PATCHES.md pointer from pyproject.
|
| |
|
|
|
|
| |
The mock terminal is a poor stand-in for a real demo; a gif/screenshots
should go there instead. The agent section pointed at .agents/, which is
no longer part of the repo.
|
| |
|
|
|
| |
Also ignore the local-only working artifacts (.auto/, .agents/, plan/,
rehearsed-engineer.md) so they stay on disk and out of the repo.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Items 4, 8 and 9 are fixed (loader switches on reopen, package exports,
py.typed), so the report and the skill no longer claim 4-9 are all open. Item 7
is partial: DatabaseManager grew a DatabaseEventCallback, but there is still
nothing an external caching client can observe, which was the point. 5 and 6
are untouched.
The "are these files byte-identical" re-check recipe is removed: it named
client.py/registry.py/resolver.py, which 0.5.x renamed, so it would have
reported "unchanged" forever. The replacement is a diff of __all__ and the
DatabaseOpenOptions fields, both of which the contract suite now asserts
whenever the library is importable.
Also records two things that cost time here: the loader-switch strip needs
input_path != source, so an .i64 passed directly alongside load options still
fatals; and an editable checkout does not enforce its own dependency floors, so
a `git pull` to 0.6.1 leaves ida-domain/zeromcp silently too old.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
0.5.x turned ida_codemode.__all__ into a real public API and hid the rest:
client.py -> handle.py, registry.py -> _registry.py + instances.py,
resolver.py -> _resolver.py. Every import we had was from a module that no
longer exists, so the TUI could not attach at all.
- handle.entry -> handle.instance (RegistryEntry -> DatabaseInstance)
- 30 keyword-only options -> DatabaseOpenOptions(...) passed as options=
- IdbBusy -> DatabaseBusyError
- InstanceDisconnected/ClientError -> DatabaseDisconnected/CodeModeConnectionError
- our scan_instances()+idb_key() ownership walks -> find_database_owner()
- our FileLock poking (_wait_for_entry_release) -> wait_database_released()
- registry.discover_instances() -> discover_databases() + InstanceState
_database_exists() is deleted with it: upstream now drops the loader switches
itself when reopening an existing IDB (_resolver._build_worker_command), which
is the same fix we had client-side. See docs/CODEMODE_UPSTREAM.md section 4 for
the one invocation that still slips through.
The offline contract suite has to keep running under a stdlib-only python3,
where every Code Mode name is bound to None -- so it now injects a strict fake
DatabaseOpenOptions and a real DatabaseBusyError exception alongside the fake
handle. Without the latter, `except DatabaseBusyError` is `except None`, and
the TypeError it raises masks whatever actually failed inside the try. The
loader-option names moved inside the options dataclass, so the guard that
caught `loading_address` vs `image_base` moved with them
(_option_fields_are_real).
uv.lock pins 0.6.1; ~/ida-venv and .venv are on 0.6.1 with the ida-domain
0.5.1 / zeromcp 1.8.0 floors it requires. Full gate: 1065 passed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Reported as "decompiler -> listing feels slow", and it was: ~900ms per Tab.
The listing view re-primes every time it is shown, and priming now builds the
whole row index. build_from_index() had no idempotence, so each switch back
re-ran segment_index over the entire segment. Measured on bash, going back to
the listing away from the primed viewport:
press Tab: 904ms -> 9.7ms, and 5 backend calls -> 0
The index is a pure function of the database and the model is thrown away and
rebuilt whenever anything moves the walk (stale_structure), so a model that is
already indexed can return immediately.
Nothing caught this because the listing was CORRECT the whole time -- only
slow. Every structural assertion passed, boot still measured fast, and the
suite has no notion of "how many calls did that keystroke cost". The new
scenario counts backend calls across three view switches and asserts
segment_index is not among them; with the guard removed again it fails.
Latent and NOT fixed here: materialising a page the viewport reaches for the
first time still happens inside render_line, i.e. an RPC (~20ms) on the UI
loop. That predates this change -- it is how skeleton pages have always
worked -- and is small enough not to read as a stall, but it is the same shape
of bug and wants prefetching onto the worker that already exists for pages.
Full gate: 1064 passed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The listing used to learn its shape by fetching it. Even after skeleton pages
that was 457 round trips and 227k rows for a 1.2MB bash, to end up knowing
how many rows there are and where each one is.
segment_index(detail=True) now returns exactly that -- every row's address,
kind and size as packed arrays, plus the page boundaries -- from one walk that
builds no rows and renders no text. ListingModel.build_from_index() decodes it
straight into _heads/_head_eas/_row_at/_by_ea/_page_*, marks every row
_SKELETON_GEN, and declares itself complete. _grow has nothing left to stream.
Nothing else in the model changed, because a row without text is a state it
already had: the FIRST read of a page materialises it through the same
_ensure_text/_ensure_page path a rename uses. That is why this is a ~90 line
change to a core view rather than a rewrite.
bash boot: 911 calls / 9.26s -> 4 calls / 1.25s 7.4x
whole census (boot + 9 UI actions): 933 calls -> 30
Two things had to be exactly right, and both are tested rather than argued:
* the ROW COUNT, or the scrollbar lies. Verified equal to a fully streamed
model, and every row's ea/kind/size equal too, 228,659 of them, zero
mismatches.
* the PAGE BOUNDARIES, or _ensure_page refetches a page that does not line
up, fails its structure check and triggers a full rebuild. heads() pages on
PHYSICAL rows; anchoring every N LOGICAL rows looks identical (the two only
diverge once a segment holds an undefined run) and would have been a
lurking bug on .bss. Anchors now carry [logical_row, ea, head_index] taken
at the real boundary, and are asserted equal to the streamer's own.
Transport note: the packed arrays are base64, not raw bytes. _PACK_EPILOGUE
serialises with json.dumps(default=str), which turns bytes into their repr --
2.97MB arrived as 11.26MB of unparseable text before that was spotted.
The new test builds both models back to back and compares every internal
array. An earlier version compared against the app's long-lived model and was
off by one row, because scenarios before it rename and define things: that
model describes the database at boot, not now.
Full gate: 1063 passed, twice.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The listing streams a whole segment for one reason -- to know how many rows
it has, so the scrollbar and paging are right. Even as skeletons that is 458
round trips and 227k rows for a 1.2MB bash, none of which is displayed.
segment_index walks the same items and counts what heads() WOULD emit,
building none of them, and returns the total plus [row, ea] anchors every 500
rows. Measured on bash, same process and database:
segment_index : 228,659 rows, 1 call, 501ms
streaming : 228,659 rows, 458 calls, 1836ms 3.7x
Exactness is the whole point, so it mirrors _rows_for's arithmetic rather
than approximating it: 3 banner rows at a function start, a label row for a
named code head that is not one, the head row, struct members for data, 2
footer rows at a function end, and an undefined run counted as its byte
length because the client presents one collapsed row as that many logical
rows. A count that is off by a handful means the scrollbar lies and a seek
lands on the wrong row, so the test compares against a fully streamed model
in the same process rather than against a tolerance, and checks that every
anchor names the address of the row it claims.
Getting that right took a false alarm worth recording: the count first looked
35 rows short of a model built by the pilot, which turned out to be a
DIFFERENT DATABASE (tests run on a pristine scratch copy). Against the same
database it matches exactly, head for head, with zero differing addresses.
Nothing consumes this yet. Spending it means teaching ListingModel to hold
sparse pages seeked through the anchors instead of one dense array grown from
the segment start, which is a real change to the core view and wants its own
run at it.
Full gate: 1054 passed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Head is the most-constructed object in the codebase -- 227k of them to stream
one bash -- and from_raw was paying for two things it did not need.
_as_int ended in a ternary whose arms were BOTH int(v, 16), so the
isinstance+startswith in front of them decided nothing and ran on every
address the client parses (55k times per 60 pages). Removed; the function is
now the isinstance it always was.
Head(...) was built by keyword, which makes the tuple match names against
fields; positional is the same object with none of that. Address conversion
is inlined for the same reason from_raw exists at all.
Together 30% off Head construction (2.27 -> 1.59ms per 2003 rows, 0.34us a
row), with output verified identical against the previous implementation over
both skeleton and full rows. isinstance was kept over the 5%-faster
"type(v) is int" because the two differ for bool and int subclasses, and that
is not a trade worth making for 5% of 12% of a page.
End to end this is ~2.5% of bash's boot (3117 -> 3065ms): Head construction
was ~12% of the client half, and the client half is ~54% of what is left.
Adds experiments/profile_client.py -- profile_remote.py's counterpart for the
half of a page load that happens outside the database process, which is where
the remaining time now is.
Full gate: 1050 passed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The listing walk asked ida_bytes.is_code/is_data ~6 times per head (the row's
kind, plus twice more via _is_unknown_f from _advance and _rows_for). Each is
a python wrapper around a C call; the mask compare is the same question with
no call at all.
Equivalence was proven, not assumed: checked against is_code/is_data over all
four classes x 16,020 synthetic flag values, and over every distinct flag in
a real database (125 across 207,191 heads). Zero mismatches. Output is
byte-identical over 3003 rows spanning all five row kinds.
(The digest DOES differ run to run, which looked alarming until it turned out
the same unmodified code differs too: hash() is seed-randomised per process
and the digest is documented as per-process. Cross-process digest comparison
proves nothing.)
Worth 5.6% on a skeleton page (3.37 -> 3.18ms) and 2.7% on a full one -- the
~5% the profile predicted, not the 1.5-2x I guessed out loud. End to end it
does NOT move bash's boot (3117 -> 3140ms, noise), because at 455 x 3.18ms
the backend is now only ~1.45s of a 3.14s boot: the other ~54% is transport,
Head construction and model bookkeeping on the client.
Only the two hot sites changed. The four in the operand-format paths are not
per-head and read better as is_code/is_data.
Full gate: 1050 passed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ListingView._grow streams the entire segment in the background for one
reason: to learn how many rows it has, so the scrollbar and paging are
right. It did that by rendering every row in full -- 227,500 rows of a
1.2MB bash, 911 backend calls, 9.3 seconds -- essentially none of which is
ever looked at.
generate_disasm_line is 22x the cost of the walk around it, so heads() gains
text=False: a SKELETON page with the same rows at the same addresses with the
same kinds and sizes, and no rendered text. Measured identical structurally
(rows, addresses, kinds, sizes and cursor all match a real page) which is
what makes one swappable for the other later. It also skips the digest
(nothing to go stale) and lets the client skip the bulk opcode read, so a
page costs ONE round trip instead of two.
Client side is deliberately tiny, because the machinery already existed: a
skeleton page is just a page whose text is stale. It is marked with a
sentinel generation no _text_gen can equal, and the FIRST read of it goes
through the same _ensure_text/_ensure_page path a rename uses -- which
already refetches a page by address, verifies the structure still lines up
and splices it in. Two staleness gates learn to fire for _skeleton as well
as _renamed; that is the whole integration.
bash boot: 911 calls / 9.26s -> 456 calls / 3.12s, 3.0x. The trade is that a
page you actually display is fetched twice (3.3ms + 9.4ms vs 9.4ms), paid
only for what is shown. _prime still loads real pages, so the viewport you
land on is never a skeleton.
The failure mode is BLANK ROWS, not an exception, and nothing in the suite
scrolled far enough to see one: _prime renders the first ~1000 rows for real,
so a test that pages down a few screens passes against a completely broken
implementation. The new scenario reads deep rows through both the model and
the render path, and asserts materialising changes neither the row count nor
the walk. Verified by reverting the two gates: it fails with text=''.
Full gate: 1050 passed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
_idatui_head_row and _idatui_line_parts were each defined TWICE at module
scope. Python keeps the last, so the first 74 lines were dead -- but the two
copies of _idatui_line_parts were not equivalent: only the live one carries
@lru_cache(maxsize=_IDATUI_LINE_CACHE).
That makes the duplication worse than clutter. Anyone tidying it up by
deleting the second copy would keep a byte-identical function and silently
lose the most important cache in the backend: 10.4us -> 3.9us per listing
row, ~2.7x on every row of every page. A comment now marks the spot and says
which copy to keep.
Found while profiling the digest; an AST duplicate-check reported the two as
IDENTICAL because ast.get_source_segment on a FunctionDef excludes the
decorator list. Compare decorators explicitly.
Verified before deleting: the block defines only names redefined below, and
nothing above it referenced them. Behaviour unchanged -- heads(500) still
9.3ms, digest still stable, and the expect/short-circuit contract still
omits rows on a match and returns them on a miss.
Full gate: 1042 passed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
_idatui_head_row ran 'import ida_bytes/ida_lines/ida_name' on every LISTING
ROW -- 25,500 sys.modules lookups per 500-row page. 50 such imports across
the file, all pointless: this module is never imported by the client
(codemode_client reads it as TEXT and installs it in the database process),
so the no-IDA house rule that forces function-local imports elsewhere does
not apply here.
Measured honestly, it is worth about 3%: heads(500) best-of-40 goes
9.37ms -> 9.07ms, median 9.73 -> 9.65. A microbenchmark of the bare import
predicted 3.2ms/page and was 10x optimistic -- the A/B is what counts, and
it is a small win, not the big one.
ida_hexrays deliberately stays function-local: it is licence-dependent, and
hoisting it would break the whole remote library for someone without the
decompiler rather than failing only when they decompile. Verified by AST
that every ida_* reference still resolves.
Adds experiments/profile_remote.py, which ships cProfile INTO the database
process -- the only way to see the split between IDA's own calls and our
python, which no client-side timer can show.
Full gate: 1042 passed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Answers 'are we batching or paying a round-trip per item' with numbers.
Wraps CodeModeClient.invoke on a live pilot and reports calls + wall + ops
for each action.
The trap it encodes: ListingView._grow streams the whole segment on a worker
thread, so it lands calls continuously regardless of what the user does. A
naive census attributes that to whichever span is open and every action comes
out at a near-identical '~1 call per 10ms of pause', which measures the pause
and not the action. It now drains the grower first, reports it as its own
line, and measures the rest against a quiet backend.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The help screen has always advertised 'PgDn / PgUp - page down / up', but
only the four code views implemented it. In the palettes the keys did
nothing at all: those screens focus a filter Input, so the OptionList's own
pageup/pagedown bindings never fire -- every key goes to the Input, and an
unhandled one is silently dropped.
Adds OptionListNav, a mixin carrying the forwarding actions, and puts the
six Input+OptionList overlays on it: symbols, search, strings, registers,
load options, projects. They already held six BYTE-IDENTICAL copies of
action_cursor_down/up, so this removes more than it adds.
Paging delegates to the widget's own action_page_up/down instead of moving
by a guessed N: those know the live viewport height, skip disabled options
and clamp at both ends -- and it keeps the forwarded panes behaving exactly
like the ones that page natively.
Two panes deliberately stay off the mixin:
* XrefsScreen focuses its list, so Textual already pages it. Now covered
by a test so nobody 'fixes' it into double-stepping.
* StructEditor binds ctrl+n to 'new type', so it cannot take NAV_BINDINGS;
it gets page actions through its existing filter-focused guard instead.
BINDINGS do not merge from a plain mixin (Textual only merges them from
DOMNode subclasses), so every screen splats *NAV_BINDINGS explicitly -- the
same trap SearchMixin documents.
Tests gate on scrollable_content_region.height >= 1 first: paging is
geometry, and before layout the page size is 0, so every check would pass
against a no-op. Verified by removing the bindings again -- 3 checks fail
with highlighted=0, which is the exact silent failure being fixed.
Full gate: 1040 passed, 0 failed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
docs/ is internal design and porting notes, not end-user documentation, so
the README should not send readers there. Put it back when there are docs
worth linking.
Removed the section and the two inline pointers that fed the same files
(-> docs/GRAPH_VIEW.md on Graph, -> docs/RPC.md on RPC), since the reason
applies to those equally. No docs/ reference remains.
Kept, deliberately, two links that are not end-user docs:
* .agents/skills/idatui/SKILL.md -- promoted from a subsection of Docs to
its own heading so removing the parent did not silently delete the only
pointer to the skill we just shipped;
* .fastfeedback/SPEED.md -- contributor-facing, sits inside Tests next to
the commands it qualifies, and is accurate.
Say the word if either should go too.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The feature coverage was right, the prose was not: eleven paragraphs that
explained MECHANISM in an intro document. The keys table right above it
already proved that scannable beats thorough here.
Every feature is now name + key + at most two lines. Longest block went from
~70 words to 35; the section from 80 lines to 53.
What got cut was rationale, not information, and none of it is lost -- each
one is already the docstring of the module that implements it: why findings
has to journal edits (journal.py), how search guesses text vs bytes
(search.py), trail painting and unseen bytes as ?? (trace.py), why terminal
graphics support is asked for rather than sniffed (kittygfx.py). The README
now says what a feature IS and links onward.
Also folded hex/strings/palettes/projects/splash into one 'Also' line (all
already in the keys table), and gave the same treatment to the Run prose and
the agent-skill paragraph.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Follows the Agent Skills standard (agentskills.io) rather than a contrib/
dir: harnesses discover .agents/skills/<name>/SKILL.md automatically from
the repo root, and it is vendor-neutral (not .claude/ or .pi/ specific).
Not a verbatim copy of the local one -- it was ~85% generic and the rest
had to go before redistributing:
* the backend is described as a normal pyproject dependency, with the
editable-git-checkout setup demoted to a conditional note (that is one
maintainer's layout, but the 'upstream pull silently swaps the backend'
warning is worth keeping for anyone who does it);
* personal agent tooling (bgrun/waitfor wrappers) replaced by the
underlying lesson -- a backgrounded subshell holds the inherited stdout,
so redirect all three fds -- which is true without those scripts;
* maintainer commit authorship dropped;
* absolute ~/dev paths made relative to the repo root; ~/ida-venv kept but
marked as this project's assumption, matching the README.
Also fixes the README's stale full-run count (788/~2m -> 1031/~50s), lists
the docs that were missing from the index, and warns that a user-level skill
of the same name shadows this one -- otherwise a personal copy silently wins
and the in-repo copy rots.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The worker-vs-Code-Mode table was measured before 0.3.2 and with both
workarounds active, so it answered a question nobody asks any more. Replaced
with three configurations measured on the same box, rolling both checkouts
back and forward:
A old client WITH workarounds on 0.3.1 -- what shipped
B current client on 0.3.1 -- what the workarounds were for
C current client on 0.3.2 -- now
Headline: the real-world gain is ~1.4x geomean, NOT the 6.9x the empty round
trip advertises, and the doc says so in those words -- because the tempting
number to quote is the wrong one. The A->C vs B->C gap is the actual story:
stock 0.3.1 was 5.4x slower, so the workarounds had already recovered nearly
everything and upstream mostly bought us the right to delete them.
Also records the three cost classes (payload- / round-trip- / IDA-dominated)
so the next person optimising here knows which lever moves which op, and the
~10% run-to-run spread so a sub-1.2x 'regression' doesn't start a hunt.
Old worker table kept below, labelled historical. experiments/bench_ops.py is
the harness, with the copy-to-/tmp-before-checkout trick documented in it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Items 1-2 were marked fixed; 3-9 were left 'not re-verified', which was a
gap rather than a finding. Checked all seven.
Item 3 is also FIXED, and by a lot: a same-box A/B (roll the editable
checkout back to 4195f21, measure, roll forward) puts
execute_python('result = 1') at 2.055ms on 0.3.1 and 0.294ms on 0.3.2 --
7.0x, now about the cost of a bare HTTP GET. The 0.3.1 column reproduces
the originally documented 2.025ms almost exactly, which is what makes the
0.3.2 column believable. So the execute_sync marshalling that was ~93% of
the floor is gone, and the 'never make a call per row' rule is much weaker.
Items 4-9 are open by construction: client/registry/resolver/server/
database/worker .py are byte-identical between 0.3.1 and 0.3.2, and all six
items live in those files. Recorded that as the cheap re-check recipe for
the next upstream pull.
SPEED.md taught the obsolete 2ms floor as a design rule; corrected, with a
warning not to go rewrite working batched code to be chatty.
|
| | |
|
| |
|
|
|
|
|
|
| |
CODEMODE_UPSTREAM items 1 (timeout_trace) and 2 (to_jsonable) both landed
upstream; note it at the top, on each item and in the priority table, and
flag that items 3-9 are not re-verified against 0.3.2. SPEED.md described
the settrace workaround and its IDATUI_CODEMODE_TRACE knob as current --
both are gone, and its backend table predates 0.3.2.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
_script() ran every snippet with sys.settrace(None) to detach Code Mode's
per-line trace hook, which returned itself and so enabled line tracing in
every frame it saw -- 52x on ida_bytes.get_flags, and the whole residual
gap against the old private worker.
0.3.2 removes that hook entirely (zero settrace refs); the deadline is now
a C-level thread interrupt, runtime._interrupt_thread. Re-measured on a
200-row listing page of main in targets/bash: the workaround buys 0.99x.
So it goes, and the caveat goes with it -- a pure-Python loop inside a
snippet is once again bounded by its deadline. _PACK_EPILOGUE measures
0.97x for the same reason (dumps_json got a C fast path) but is kept for
encoder determinism, with its stale 114x claim corrected.
experiments/bench_pack_trace.py is the harness for both numbers.
Full gate green: 1031 passed, 0 failed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
"EMPTY BL" in the status bar is triskel's own bracket-list assertion from
its SESE pass, and it turned out to be the mild version of the problem.
Triskel's graph root is whichever node was created FIRST, and every
analysis walks out from it. Anything unreachable from that node is
undefined behaviour. We were:
- creating nodes in id order, so the root was the lowest-numbered
block rather than the entry, and
- splitting only WEAKLY connected components, which says nothing about
reachability.
A 7-block CFG whose entry has no successors -- IDA hands those out for
thunks and for dead code it could not resolve -- SEGFAULTS the
interpreter. That is unsurvivable: it takes the session down and there
is no exception to fall back from.
Now the entry is created first, orphan blocks are attached to it with
phantom edges that steer placement but are never drawn (one edge usually
adopts a whole orphan subgraph, attached at a node no other orphan
reaches), and reachability is asserted in python BEFORE crossing into
C++. This replaces the component splitting entirely: one layout instead
of N stacked side by side, and triskel gets to place the orphans.
The reproducer is now a test (t_unreachable_entry). Remaining fallbacks
on the ls corpus are 8/1200 layouts, all the upstream box-overlap
defect, all but one on 300-500 block functions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
"engine triskel -> native+triskel-failed" was unreadable and, worse,
gave no reason: the cause went to a logger a TUI user never sees.
Dumped all 400 functions of bin/ls (echo's 128 were not enough) and swept
them at three zoom levels: 6 of 1200 layouts fell back, all of them my
own _verify tripping over a detour that could not be placed.
- the detour jumped to the nearest side of the FIRST box in the way,
which in a dense layout is usually inside the next box along. It now
collects every box the run passes and picks the nearest genuinely
free line.
- it skipped the first and last segments because they carry the port
and the arrowhead. But that is exactly where the failures were:
triskel is happy to park a block directly above its successor and
drive the final approach straight through it. Those segments may now
move ALONG their own box's border, which is free almost every time.
- repairs are swept to a fixed point: moving one segment stretches its
neighbours, which can push those into a box.
0/1200 fallbacks after that. Then the bigger corpus turned up a second,
genuinely upstream defect: superimposing SESE regions can leave two
blocks a couple of columns into each other (2 of ls's 400 functions,
in float space, before rounding). Cosmetic in a PNG; here the boxes are
made of text, so one block's disassembly overwrites another's. _verify
now checks it and falls back, which is the right trade.
Reporting, so this is never mute again:
- stats["engine_error"] carries the reason, the status line shows it,
and the label is "native (triskel failed)".
- the corpus test asserts fallbacks are rare AND explained, rather
than asserting they never happen.
Also lowered AUTO_TRISKEL_MAX_BLOCKS 250 -> 180. Layout runs on every
zoom keypress and triskel knees hard past ~175 blocks (174: 66ms,
233: 489ms, 329: 555ms). The old cap allowed a 489ms stall. The corpus
timing check now measures only sizes `auto` can actually reach, plus a
5s ceiling so nothing blows up quadratically when forced.
|
| |
|
|
|
|
|
|
|
| |
./ida-tui runs $IDATUI_PYTHON (default ~/ida-venv), which is not the
repo .venv the tests use -- so the graph view can report pytriskel as
missing while tests/test_graph.py is exercising both engines. "not
installed" on its own sends you to check the wrong python; the status
now names sys.executable, and docs/GRAPH_VIEW.md says to install into
both.
|
| | |
|
