| Commit message (Collapse) | Author | Files | Lines |
|---|
|
Reported as "decompiler -> listing feels slow", and it was: ~900ms per Tab.
The listing view re-primes every time it is shown, and priming now builds the
whole row index. build_from_index() had no idempotence, so each switch back
re-ran segment_index over the entire segment. Measured on bash, going back to
the listing away from the primed viewport:
press Tab: 904ms -> 9.7ms, and 5 backend calls -> 0
The index is a pure function of the database and the model is thrown away and
rebuilt whenever anything moves the walk (stale_structure), so a model that is
already indexed can return immediately.
Nothing caught this because the listing was CORRECT the whole time -- only
slow. Every structural assertion passed, boot still measured fast, and the
suite has no notion of "how many calls did that keystroke cost". The new
scenario counts backend calls across three view switches and asserts
segment_index is not among them; with the guard removed again it fails.
Latent and NOT fixed here: materialising a page the viewport reaches for the
first time still happens inside render_line, i.e. an RPC (~20ms) on the UI
loop. That predates this change -- it is how skeleton pages have always
worked -- and is small enough not to read as a stall, but it is the same shape
of bug and wants prefetching onto the worker that already exists for pages.
Full gate: 1064 passed.
|
|
The listing used to learn its shape by fetching it. Even after skeleton pages
that was 457 round trips and 227k rows for a 1.2MB bash, to end up knowing
how many rows there are and where each one is.
segment_index(detail=True) now returns exactly that -- every row's address,
kind and size as packed arrays, plus the page boundaries -- from one walk that
builds no rows and renders no text. ListingModel.build_from_index() decodes it
straight into _heads/_head_eas/_row_at/_by_ea/_page_*, marks every row
_SKELETON_GEN, and declares itself complete. _grow has nothing left to stream.
Nothing else in the model changed, because a row without text is a state it
already had: the FIRST read of a page materialises it through the same
_ensure_text/_ensure_page path a rename uses. That is why this is a ~90 line
change to a core view rather than a rewrite.
bash boot: 911 calls / 9.26s -> 4 calls / 1.25s 7.4x
whole census (boot + 9 UI actions): 933 calls -> 30
Two things had to be exactly right, and both are tested rather than argued:
* the ROW COUNT, or the scrollbar lies. Verified equal to a fully streamed
model, and every row's ea/kind/size equal too, 228,659 of them, zero
mismatches.
* the PAGE BOUNDARIES, or _ensure_page refetches a page that does not line
up, fails its structure check and triggers a full rebuild. heads() pages on
PHYSICAL rows; anchoring every N LOGICAL rows looks identical (the two only
diverge once a segment holds an undefined run) and would have been a
lurking bug on .bss. Anchors now carry [logical_row, ea, head_index] taken
at the real boundary, and are asserted equal to the streamer's own.
Transport note: the packed arrays are base64, not raw bytes. _PACK_EPILOGUE
serialises with json.dumps(default=str), which turns bytes into their repr --
2.97MB arrived as 11.26MB of unparseable text before that was spotted.
The new test builds both models back to back and compares every internal
array. An earlier version compared against the app's long-lived model and was
off by one row, because scenarios before it rename and define things: that
model describes the database at boot, not now.
Full gate: 1063 passed, twice.
|
|
The listing streams a whole segment for one reason -- to know how many rows
it has, so the scrollbar and paging are right. Even as skeletons that is 458
round trips and 227k rows for a 1.2MB bash, none of which is displayed.
segment_index walks the same items and counts what heads() WOULD emit,
building none of them, and returns the total plus [row, ea] anchors every 500
rows. Measured on bash, same process and database:
segment_index : 228,659 rows, 1 call, 501ms
streaming : 228,659 rows, 458 calls, 1836ms 3.7x
Exactness is the whole point, so it mirrors _rows_for's arithmetic rather
than approximating it: 3 banner rows at a function start, a label row for a
named code head that is not one, the head row, struct members for data, 2
footer rows at a function end, and an undefined run counted as its byte
length because the client presents one collapsed row as that many logical
rows. A count that is off by a handful means the scrollbar lies and a seek
lands on the wrong row, so the test compares against a fully streamed model
in the same process rather than against a tolerance, and checks that every
anchor names the address of the row it claims.
Getting that right took a false alarm worth recording: the count first looked
35 rows short of a model built by the pilot, which turned out to be a
DIFFERENT DATABASE (tests run on a pristine scratch copy). Against the same
database it matches exactly, head for head, with zero differing addresses.
Nothing consumes this yet. Spending it means teaching ListingModel to hold
sparse pages seeked through the anchors instead of one dense array grown from
the segment start, which is a real change to the core view and wants its own
run at it.
Full gate: 1054 passed.
|
|
ListingView._grow streams the entire segment in the background for one
reason: to learn how many rows it has, so the scrollbar and paging are
right. It did that by rendering every row in full -- 227,500 rows of a
1.2MB bash, 911 backend calls, 9.3 seconds -- essentially none of which is
ever looked at.
generate_disasm_line is 22x the cost of the walk around it, so heads() gains
text=False: a SKELETON page with the same rows at the same addresses with the
same kinds and sizes, and no rendered text. Measured identical structurally
(rows, addresses, kinds, sizes and cursor all match a real page) which is
what makes one swappable for the other later. It also skips the digest
(nothing to go stale) and lets the client skip the bulk opcode read, so a
page costs ONE round trip instead of two.
Client side is deliberately tiny, because the machinery already existed: a
skeleton page is just a page whose text is stale. It is marked with a
sentinel generation no _text_gen can equal, and the FIRST read of it goes
through the same _ensure_text/_ensure_page path a rename uses -- which
already refetches a page by address, verifies the structure still lines up
and splices it in. Two staleness gates learn to fire for _skeleton as well
as _renamed; that is the whole integration.
bash boot: 911 calls / 9.26s -> 456 calls / 3.12s, 3.0x. The trade is that a
page you actually display is fetched twice (3.3ms + 9.4ms vs 9.4ms), paid
only for what is shown. _prime still loads real pages, so the viewport you
land on is never a skeleton.
The failure mode is BLANK ROWS, not an exception, and nothing in the suite
scrolled far enough to see one: _prime renders the first ~1000 rows for real,
so a test that pages down a few screens passes against a completely broken
implementation. The new scenario reads deep rows through both the model and
the render path, and asserts materialising changes neither the row count nor
the walk. Verified by reverting the two gates: it fails with text=''.
Full gate: 1050 passed.
|
|
The help screen has always advertised 'PgDn / PgUp - page down / up', but
only the four code views implemented it. In the palettes the keys did
nothing at all: those screens focus a filter Input, so the OptionList's own
pageup/pagedown bindings never fire -- every key goes to the Input, and an
unhandled one is silently dropped.
Adds OptionListNav, a mixin carrying the forwarding actions, and puts the
six Input+OptionList overlays on it: symbols, search, strings, registers,
load options, projects. They already held six BYTE-IDENTICAL copies of
action_cursor_down/up, so this removes more than it adds.
Paging delegates to the widget's own action_page_up/down instead of moving
by a guessed N: those know the live viewport height, skip disabled options
and clamp at both ends -- and it keeps the forwarded panes behaving exactly
like the ones that page natively.
Two panes deliberately stay off the mixin:
* XrefsScreen focuses its list, so Textual already pages it. Now covered
by a test so nobody 'fixes' it into double-stepping.
* StructEditor binds ctrl+n to 'new type', so it cannot take NAV_BINDINGS;
it gets page actions through its existing filter-focused guard instead.
BINDINGS do not merge from a plain mixin (Textual only merges them from
DOMNode subclasses), so every screen splats *NAV_BINDINGS explicitly -- the
same trap SearchMixin documents.
Tests gate on scrollable_content_region.height >= 1 first: paging is
geometry, and before layout the page size is 0, so every check would pass
against a no-op. Verified by removing the bindings again -- 3 checks fail
with highlighted=0, which is the exact silent failure being fixed.
Full gate: 1040 passed, 0 failed.
|
|
`e` in graph mode cycles auto -> native -> triskel, and `auto` prefers
triskel where it is installed and the function is at most 250 blocks.
Why: our layered engine draws wide-and-short pictures with a lot of
crossings on anything branchy. Triskel splits the CFG into Single-Entry
Single-Exit regions first and lays each out on its own, which on the
128-function corpus means fewer crossings on 12 functions, equal on 9,
worse on 3 -- and the wins are the hairballs (sub_5CA0 41 -> 6,
sub_2C90 32 -> 7, sub_2C00 12 -> 0). It also routes loop edges around
the side of the graph the way IDA does, which was a known gap here.
It is not free: ~2x slower at 87 blocks, 10x at 424, hence the cap.
The library needed a fork (~/dev/triskel, branch idatui) before it could
be used from Python at all -- its get_waypoints() threw on every
published version, an empty graph segfaulted the interpreter, and its
spacing constants were pixels baked in at compile time. Making those
settable is what makes this integration cheap: we hand it CELLS, so
its output is integral and two edge lanes can never round onto the same
row. The feared quantisation problem measured out backwards -- cells
claimed by more than one edge: native 131, triskel 35.
Not trusted with degenerate input, all handled before the call:
self-loops and disconnected components make it throw, and one corpus
edge comes back routed through a block, which we detour and re-verify.
A triskel failure is never fatal; it falls back to native.
Two things the second engine flushed out of the existing code:
- the canvas was sized from boxes alone, which is exact only because
native's dummy nodes reserve the space. Triskel routes outside that
bounding box and the edges were being clipped.
- arrowhead placement read e.back, conflating "this is a loop edge"
(style) with "this polyline runs against control flow" (geometry).
Now Edge.flipped, which is also a latent fix for residual-cycle edges
whose succ/pred were being reported backwards.
tests/test_graph.py runs its whole suite once per available engine
(943 checks); new graph_engine scenario covers the live toggle.
|
|
Reported as "the splash logo stopped rendering". It had not stopped: the
splash asks for the artwork's NATURAL size and shows nothing when that
does not fit, and the artwork needs 31 rows plus 10 of box chrome. A pane
in a split zellij window is 31 rows — one row short of the 41 it wanted —
so the logo silently disappeared. Traced with $IDATUI_KITTY_LOG in the
real session:
compose: supported=True app.size=Size(width=159, height=31)
cells=60x23 fits=False
The terminal scales an image into whatever cell box it is placed in
(`c=`/`r=` on the placement), so there was never a reason for
all-or-nothing. `logo_cells(max_rows)` now fits the art to the room left
after the box's furniture, and the same number reserves the cells and
sizes the placement, so a resize needs no relayout. In that same 31-row
pane it now draws 55x21 instead of nothing.
Two things fixed on the way:
* The chrome constant was one row optimistic (`rows + 9` where the box
measures 10: border 2, padding 2, art margin 1, title 1, note 1+1,
help 1+1). At exactly the old threshold the help line was clipped off
the bottom.
* `_fits` conflated "is the terminal big enough" with "is the artwork the
right size", which is what made the image path inherit the block art's
all-or-nothing behaviour. The block art genuinely cannot scale (it is
half-block cells, 26 rows) and still falls back to the text splash;
the image no longer does.
`splash_scaling` pins it at 31, 30 and 44 rows: the logo is drawn, it is
scaled to the room, the box is never clipped, and a big pane still gets
the natural size.
905 passed, 0 failed.
|
|
SearchPalette opened pinned to the top of the screen: the CSS named the
screens that centre (`SymbolPalette, StringsPalette, ProjectPalette, …`)
and a new dialog is not on a list nobody remembers to edit. The comment
sitting above that rule — "every #pal-box palette centres, not just the
symbol one" — was the *first* time this happened.
`ModalScreen { align: center middle; }` matches subclasses, so every
dialog inherits it and the next one is centred for free; the eight
per-screen rules that only repeated it are gone. Textual's own Ctrl+P
CommandPalette is a ModalScreen too and wants its stock top alignment, so
it opts out in one visible line rather than by omission.
The `modal_centering` scenario checks both halves: that centring is
expressed as a rule, and that it actually reaches a dialog's laid-out
region (above/below and left/right within a cell).
894 passed, 0 failed.
|
|
`/` only ever searched the lines of the view you were in. This adds the
search you actually need on a binary: over the entire database, either
through the rendered disassembly or through the image.
* **text** matches the line as displayed, whitespace-normalised, so
`call cs:` finds `call cs:getenv_ptr` (IDA's column padding is not
something anyone types). Smartcase; `regex` available over RPC.
* **bytes** is IDA's own `find_bytes`, so the pattern language people
already know works unchanged: hex pairs, `?` wildcards for a whole byte
or one nibble (`48 8? ?? 24`), quoted literals (`"Hello", 0`). Commas,
no separators (`488B05C3`) and ragged spacing all normalise.
**Which mode you meant is guessed, and the guess is biased on purpose.**
`dead`, `add`, `cafe` and `ff` are valid hex AND ordinary things to search
for, so a bare hex-looking word stays TEXT; nobody types `48 8b ?? c3`
meaning prose. `hex:`/`text:` prefixes and F2 override it.
The subtle case is a *typo* in a byte pattern. `48 zz c3` first fell
through to a text search and reported "no match" — indistinguishable from
"those bytes are not in this binary", which is the most misleading answer
a search can give. Now any query whose tokens are all byte-sized is
treated as bytes, and a bad token is refused BY NAME. IDA does the same
thing quietly (find_bytes answers a malformed pattern with zero hits and
no error), so the validation lives in Program.search, not just in the UI.
Enter searches, then Enter opens the highlighted hit; the title says which
it will do, because a database-wide scan is far too slow to run on every
keystroke like the other palettes. Navigation goes to the item head — a
byte match can start mid-instruction — and the status names the exact
address.
Also: the `find` RPC verb and `drive find`, which is the one an agent
wants (`drive find '48 8b ?? c3'`).
idatui/search.py holds the classification and is pure, so the whole
question of "what did they mean" is tested offline: tests/test_search.py,
35 checks, 0.1s. Pilot scenario db_search covers the UI end to end.
Full suite: 890 passed, 0 failed, 51.3s.
|
|
The output of an RE session is what you worked out, and it was locked in a
.i64 that only IDA can read. Ctrl+E (or `drive export`, or the `export` RPC
verb) writes it out: your comments grouped by function with the line each
annotates, the names and prototypes you set, the types you declared.
**The hard part was provenance, and it needed a mechanism, not a filter.**
A database does not record WHO wrote a comment or a name. IDA's analyzer
sets `; switch 73 cases` and `; s1` with the same `set_cmt` a person uses,
and the ELF loader sets `elf_gnu_hash_nbuckets` and `File class: 64-bit`
the same way. Four probes, all negative: the FF_COMM flag is identical,
`get_cmt` returns them all, `generate_disasm_line` tags every one of them
COLOR_REGCMT (not COLOR_AUTOCMT), and they survive with auto-comments
switched off. A first cut filtered by shape and produced a report whose
first screen was ELF header trivia and `; jumptable ... case 99`.
So idatui journals its own edits (idatui/journal.py) into a netnode in the
database: it rides along in the .i64, it is still there next session, and
the report is then exactly what was done here -- 2 findings out of a
database carrying 693 other annotations. Recorded at the choke points in
edit_ctl (rename, name-address, comment, retype) and in the struct editor;
flushed on save, on export and on quit, so no edit pays a round trip.
Without a journal (a database worked on in the IDA GUI, or predating this)
the report falls back to filtering by shape -- dummy names, imports, loader
segments, the analyzer's stereotyped switch/jumptable strings -- and says
so in the document rather than claiming authorship it cannot prove.
idatui/findings.py splits gather (needs IDA) from render (does not), so the
formatting, grouping, sorting, escaping and the empty cases are tested
offline: tests/test_findings.py, 32 checks, no worker, 0.1s. The pilot
scenario covers the round trip that matters -- edit through the UI, export,
find it in the file, and reload the journal from the .i64.
Full suite: 842 passed, 0 failed, 51.2s.
|
|
The suite spent its time in two kinds of guess.
**Flat pauses.** ~140 `pause(d)` calls were 20.4s of the pilot's 62s, and
`test_trace_ui` was 13.5s of `pilot.pause(1.0)` out of 19.6s. `Ctx.pause` is
now `settle` (`d` is the upper bound, not the cost) and the other suites'
sleeps became gates on the thing the check is about. `Ctx.sleep` stays for
what a timer really drives.
**Textual's keypress path.** `Pilot.press` calls `wait_for_idle` twice per
key, which sleeps in 20ms granules until process time stops advancing --
84ms per keypress here, 23s of the pilot's 43s. `_fixtures.fast_keys()`
replaces it with the gate the suites already use: send the keys, then
settle. Deleting the heuristic *without* that broke nine checks, so it was
doing a job, badly.
Four checks turned out to be riding on those sleeps: they read geometry or
a repaint (`si.region`, `gv._minimap_rect()`, glyphs off `gv.render_line`,
a repaint trace), and a settled app has not necessarily been laid out or
painted. They now wait for the frame. The debounced function filter
(`set_timer(0.08)`) likewise waits for its effect.
Also fixed two waits on signals that never arrive: the comment wait in
`rename` carried a `dec.loaded_ea == app._cur.ea` conjunct that cost 9s of
timeout and then let the check pass vacuously, and `listing_view` -- the
one entry under "Known-flaky" -- waited on `lst.total`, which is true
before a single row exists.
`--profile` reports, per scenario, seconds settling / waiting / pressing,
and names any wait that expired with its line number. It is how the above
was found and how the next 20s should be.
Verified: 4 full `tests/run.py` runs, 800 passed each, 49.0-49.2s
(was 117.4s); 4 consecutive pilot runs, 313 passed each, 21.2s (was 63.7s).
|
|
|
|
|
|
Mechanical part of the port: the 27-file patch was cut against a base ~148
commits behind us, so it did not apply. Resolved 11 conflicts (all of them
diff drift, not semantic clashes) and the three file deletions:
- app.py: the patch re-inserted _do_rename/_do_name_addr/_seek_split etc. as
"theirs" because our tree moved them to edit_ctl.py/trace_ctl.py. Kept ours
and applied the real intent (WorkerClient->CodeModeClient, .call->.invoke,
_open_worker_client->_open_database_client) at their current homes.
- domain.py: kept Head as a NamedTuple -- the patch reverted it to a frozen
dataclass, which the perf work measured at 2.9us vs 1.9us per row on a
quarter-million-row walk. Dropped _fetch_output (no download_url under Code
Mode) and its now-dead urllib/json imports.
- pane.py: the patch's deletion swallowed our zellij support along with the
worker-reaping block it meant to remove. Kept zellij, removed the reaping.
- test_scenarios.py: the idb_save->save_database teardown hunk belongs to
tests/_fixtures.py in our tree; applied it there and kept our pc_num_format
scenario that the drift landed on.
Three defects in the patch itself, fixed here:
- It made "import idatui" hard-require ida_codemode, so every offline suite
died at import -- including the pure ones (graph/index/trace) that are the
house rule for "tests/run.py --fast". The import is now deferred and gated
on the binding, which is also what lets the port's own contract tests
inject a fake DatabaseHandle.
- project.stage() inlined an ida_codemode.registry import and treated "library
not installed" as "someone owns this database", which broke IDA-free project
staging. Ownership lookup moved to codemode_client.database_owner().
- tests/test_codemode_client.py had no NEEDS_IDA marker, which tests/run.py
rejects outright.
Offline suite: 301 passed, 0 failed. Against master's 344 the whole delta is
accounted for: -40 worker_client (module deleted), -18 launch sweep checks
(behaviour deliberately removed) +3 guarding that it stays removed, +2 pool
(GUI-save semantics), +13 new codemode_client contract tests.
NOT yet done, and the port is not functional without it: the adapter is
missing five operations our tree grew since the patch's base (flowchart,
op_format, pc_nums, pc_num_format, survey_binary) and its "heads" predates
back-walking and digest/expect.
|
|
opcode bytes already attached) with the graph_minimap scenario's racy SETUP made deterministic: clear _graph_sticky before the second navigation so Space is known to be entering the graph, not leaving it. No assertion changed.
Result: {"status":"keep","total_ms":22980.2,"lg_boot_ms":738.2,"lg_decomp_ms":2401.8,"lg_graph_ms":944.1,"lg_hex_ms":920.6,"lg_index_ms":75.2,"lg_listing_cold_ms":538.5,"lg_listing_warm_ms":411.1,"lg_nav_ms":6813.9,"lg_palette_ms":4.9,"lg_render_ms":221.8,"lg_search_ms":5630.1,"pure_graph_ms":240.7,"sm_boot_ms":537.5,"sm_decomp_ms":595.1,"sm_graph_ms":715.7,"sm_hex_ms":858.8,"sm_index_ms":0,"sm_listing_cold_ms":263.3,"sm_listing_warm_ms":265.3,"sm_nav_ms":335.2,"sm_palette_ms":0.3,"sm_render_ms":271.4,"sm_search_ms":196.5,"fails":0}
|
|
A TUI must not die because one background load failed, so this codebase catches
broadly -- ~50 `except Exception` sites, two dozen resolving to `pass`. Right
policy, one bad consequence: with 44 `@work(thread=True)` workers, a failure in
a background load leaves no trace whatsoever. The view stays empty and there is
nothing to read afterwards, because the app owns the screen.
kittygfx already solved this for itself with $IDATUI_KITTY_LOG. idatui/diag.py
is the same idea for everything else: $IDATUI_LOG writes every swallowed error
plus its traceback to a file, and the last 50 are kept in memory regardless so a
driver can ask a live app what went wrong. Unset, it costs an environ lookup.
Wired in where losing the error changes a DECISION rather than just a pixel:
* rename: a resolve() that throws renames as DATA instead of as a function.
* name: a function_of() that throws means we never learn the address is a
function start, so the index keeps the old name and every readback says the
rename didn't happen.
* retype: a resolve() that throws retypes the ENCLOSING function instead.
* decompile: a failed full-body fetch silently returns CLIPPED pseudocode.
* trail: a failed decomp_map stops the pseudocode being painted, silently.
Deliberately NOT wired into the query_one guards -- a modal owning the screen is
normal and constant, and logging it would bury the real entries in noise.
New RPC verb `diag {n?, clear?}`, documented in docs/RPC.md: the answer to "the
verb reported success and the pane shows nothing".
Also a flake, same shape as the others: follow_xrefs waited on the nav depth but
asserted on _cur, and a follow pushes the source entry BEFORE opening the
target -- so the check could run in between and see the function it jumped
from. About one run in ten. It waits on the postcondition it asserts now; three
clean full runs since.
833 checks; --fast is 344 in 3.5s.
|
|
_active was a bare string with 49 comparisons across four modules and a fifth
value nobody meant to keep. "disasm" was assigned on exactly one path -- a
decompile that failed with nowhere to return to -- and named the same widget as
"listing". Four sites understood it; five compared against "listing" alone and
silently took the wrong branch:
* Tab out of a failed decompile set "listing" instead of "decomp", so the
first press appeared to do nothing.
* rpc.py carried a workaround for a mode change that never arrived, keyed on
being ALREADY in the ghost state -- so it fired in the rare case and not in
the common one. Now keyed on LISTING, which is the case that happens.
* drive.py asked the socket to show it "disasm", a value the app will now
never report, and would have toggled twice and given up.
ViewMode is a StrEnum on purpose: _active goes straight to drivers as
cursor.kind and the pilot compares it to plain strings, so members being strings
keeps every payload and comparison working. What it buys is one place that says
which modes exist, and an AttributeError instead of silence on a typo.
Read it through is_listing/is_decomp/is_hex/is_graph/in_code rather than ==.
The bare comparisons are what let the ghost hide, and they are what the next
mode would have to hunt down -- adding "graph" already cost one crash that way
(_active_code_view returning None when a prompt closed).
view_modes_all_handled walks the enum and asks the app the questions it asks
itself. Verified it bites: adding a fifth unhandled member fails it twice.
746 checks, 142.3s.
|
|
The split resync loop (f898350) had no test. Two attempts at one were worthless
and are not in this commit: a scroll-based guard passed with the bug
reintroduced, and a constructed anchor -- inside the loaded function, outside
its mapped span -- skipped, because on this target the map covers the whole
function. The real trigger is the race window while the decomp map lags the
decompiler re-pointing, which is tedious to force but wide open in split_view's
own flow.
So split_view counts lookup_funcs across its body and bounds it. Verified both
ways, which is the only reason it's worth having: 29,227 calls with the bug put
back, under 500 with the fix. The bound is loose because the bug was three
orders of magnitude out, not a near miss.
733 checks, 139s.
|
|
all_funcs() forced a full load of the function index only when it was EMPTY, so
a partially streamed index -- non-empty but incomplete, which is exactly the
state during boot and after any bump_items() -- came back truncated. Every
fixture picked through find_func/biggest therefore depended on how far streaming
had got by the time a scenario asked.
That is the graph_minimap flake: on an unlucky run find_func(size > 0x300)
picked a much larger function than usual, whose graph never finished inside the
scenario's own 60s wait. Three failures and 65 seconds, one run in several, with
no code change to blame. Three consecutive clean runs at 1.7s since.
CORRECTION to f898350, which said a range cache for function_of 'broke
graph_minimap'. It did not. The failure happened in the run after I added the
cache and I attributed it without checking; it recurred with the cache long
gone. The cache is still not here, but for the honest reason: with the resync
loop fixed, function_of is down to 340 calls and 1.4s across the whole suite,
so caching it is not worth the invalidation surface.
Suite 195.7s -> 138.4s, 732 checks.
|
|
test_trace_ui spent 18.8 of its 35.2 seconds in flat pilot.pause() calls placed
to let an async seek land. Two loops were most of it: 6 iterations at 0.5s and
28 at 0.3s, 11.4s of sleeping to check that a step moves the cursor.
They are condition waits now. The questions are unchanged -- does the listing
cursor reach the pc, does the pseudocode cursor follow -- but they cost what
they cost instead of a fixed budget. The second loop settles on something that
does NOT presuppose the answer (the listing cursor arriving, and the trail map
belonging to the loaded function): waiting on 'is this pc mapped' would have
burned the timeout on every unmapped instruction, about half of them, and come
out slower than the sleep it replaced.
35.2s -> 20.9s, 39 checks, stable over repeated runs.
tests/_fixtures.py collects the staging both this suite and test_scenarios need
-- scratch copy, seeded from a golden .i64 nothing writes back to -- which was
private to test_scenarios. Worth saying plainly: on targets/echo the seeding is
worth 0.19s, not the analysis time I assumed when I went looking. It is shared
for the deduplication and for whatever gets pointed at a bigger binary.
|
|
_split_range is the min/max of the decomp_map's addresses, which does not cover
every address in the function -- Hex-Rays doesn't attribute them all. An anchor
inside the loaded function but outside that span therefore asked _sync_split for
a resync, _apply_resync found the function already decompiled, called
_sync_split again, and it asked again. One thread worker and one lookup_funcs
round trip per iteration, for as long as the cursor sat there.
Measured in the pilot: 23,888 function_of calls in one scenario across FOUR
distinct addresses, 21,156 of them for 0x2060 alone. In the live app that is an
idle split view pegging the worker.
_sync_split grows a resync flag; the one caller that is itself the resync passes
resync=False, so the branch can be entered at most once per chain.
While measuring, three scenarios waited on "fail" appearing in the status --
the app says "cannot decompile". decomp_fallback burned its full 25s timeout and
then passed a check on _active == "listing", which was already true before Tab
was pressed: it asserted nothing, slowly. Now waits for the real text and checks
that the fallback actually said something.
scenarios 115.8s -> 74.9s, suite 195.7s -> 153.3s, 732 checks green.
Not included: a range cache for function_of. It broke graph_minimap (the graph
stopped loading at all -- the 65s was that scenario's own 60s wait timing out)
and with the loop gone it buys little. Left out rather than shipped
half-understood.
|
|
Fourteen test files, each its own __main__, and no way to run them but from
memory -- so in practice you ran the one you were working on and hoped. Worse,
nothing said which files need a licensed IDA and a real worker (minutes) and
which are pure stdlib (milliseconds), so the cheap ones nobody ran either.
tests/run.py runs the lot and prints one table. --fast selects only the suites
that need nothing, which is 257 checks in half a second under any python3 --
that's the one you run between edits.
The classification lives in the test files, not in a table here that would rot
the first time someone adds a test: each declares NEEDS_IDA at module scope and
run.py reads it with ast (it can't import them -- they run their suite at
import). A file without the marker is a hard error rather than a silent guess.
|
|
F1 is swallowed before it ever reaches us on at least one setup here -- the
app's own binding fires when the key is injected directly, zellij has no F1
binding of its own, and every common F1 encoding written straight into the pane
(SS3 ESC O P, CSI ESC [11~, CSI-u ESC [1;1P) opens it. So the key is being eaten
by something upstream, which is not ours to fix, and a cheatsheet reachable only
through a function key is fragile anyway: terminals and multiplexers claim them
routinely.
H opens and closes it too. '?' stays with the incremental search, which is what
it has always done in the code views.
|
|
Clicking the minimap panned to the exact coordinate under the pointer and moved
the cursor only if a block happened to sit there. Since one minimap cell covers
many canvas cells, "there" was almost always padding: you got a jump into empty
space and the cursor stayed behind, so you had to click a block afterwards to
actually go anywhere.
Blocks cover a few percent of a laid-out graph -- 4.6% of an 87-block function,
0.8% of a 424-block one -- and the rest is the space that keeps edges apart. So
coordinates are the wrong thing to navigate by here. The minimap now snaps to
the nearest block and takes the cursor with it, and a drag scrubs from block to
block. Distance is measured with the column halved, because cells are twice as
tall as they are wide and otherwise "nearest" is not what looks nearest.
A drag-pan or ctrl+d/pageup that ends with no block on screen at all now eases
to the nearest one too, since an empty screen leaves nothing to navigate back
by. It only fires when nothing is visible, so a deliberate pan is never fought.
|
|
Click it to jump the view to that part of the graph, drag to scrub. If the
point you clicked is over a block the cursor lands in it, so the keyboard
carries on from where you pointed instead of snapping back.
This also fixes a real bug rather than only adding a feature. The minimap
FLOATS over the canvas -- it is pinned to the viewport, not drawn into the
graph -- so a click on it was being translated into canvas coordinates and
dropping the cursor into whatever block happened to lie underneath. It has to
be hit-tested before the canvas, which is what on_click now does.
_minimap_rect() is the one source of truth for where it is: the renderer and
the hit-test both take the position from it, so the two-column inset that
keeps it clear of the ScrollView's scrollbar can't drift between them.
|
|
Opening, navigation and edge-following, the three zoom levels, the drawing
actually reaching the screen (a layout that is right but paints nothing
looks fine from the outside), clicking a block, renaming from inside one,
and the mode surviving a navigation. The help test now derives its group
list from _HELP instead of hardcoding it, so adding a card isn't a failure.
|
|
Pilot scenarios for the listing and the pseudocode, for the mark moving
between operands, for a refusal not being swallowed by the previous
success, and for the cursor staying on its literal across a reflow. Plus
experiments/opfmt_tools.py, which runs the real injected tool sources
against a live database with the decorators stubbed -- faster than the
pilot and the right place for the IDA-side edge cases.
Also fixes two pre-existing bugs the work surfaced, both of which made
edits happen off screen: cursor_on searched from row 0 of the whole
segment and never scrolled, so a driver's word= edit landed in an
unrelated function while reporting success; and the cursor verb didn't
scroll either. Both now go through rpc.place_cursor.
|
|
The status bar said "0x2490" without saying what it belonged to. Obvious once
there are two panes open, or after switching binaries in a project — which
already prefixed its label, so single-binary sessions were the odd ones out.
[cat] .text @ 0x472b [listing] (c code · p func · u undefine · Enter follow)
Uses the opened file's basename, not _module(): that one asks the worker over
RPC and this runs on every status write. Kept in step when the path changes
(project switch, reload).
Three messages already carried the module name themselves and would have read
"[echo] echo — 128 functions"; they don't say it twice now.
tests: +3 scenarios (212) — the bar names the file, keeps naming it as you move
(the idle status is not the only writer), and doesn't say it twice.
|
|
The suite edits the database — defines code, undefines items, renames, comments —
and IDA saves all of it. Running that against targets/echo.i64 meant every run
inherited the last one's damage.
That cost real time twice. decomp_follow_self "started failing" with no code
change, and stayed failing until the .i64 was deleted; an edit-position check
looked flaky about one run in three and I nearly reported it as an async race.
Both were the database drifting. A suite whose result depends on its own history
cannot be trusted to accuse the code — and it had been quietly laundering bad
conclusions for however long.
Now the suite copies the binary into a temp dir and seeds it from a golden
database (<target>.pristine.i64) that nothing ever writes back to. Every run
starts from identical bytes; the tracked target is never opened.
The golden copy is built once, on first run, by analysing and saving before any
scenario runs — so it costs one analysis rather than one per run. Rebuilt
automatically if the binary is newer.
Verified: two consecutive full runs both 209/0; targets/echo.i64 no longer
exists after a run; a deliberately corrupted targets/echo.i64 is ignored
completely (11/0 with junk in place, and the junk untouched afterwards); no temp
directories leak.
The other suites were already clean for the same reason, by different means:
test_blob_ui builds a throwaway binary, test_project_ui stages copies, and
test_thumb_ui deletes the .i64 before each phase because the T flag and the
segment's bitness are saved in it.
|
|
DisasmView was the function-scoped code view from before the unification. The
app hasn't instantiated it since — compose() yields only ListingView, and even
the test harness's Ctx.dis returns ListingView with a comment saying so. Its
CursorMoved messages had no handler, so every one it posted went nowhere.
304 lines of rendering, search, cursor and navigation logic that never ran. It
also cost real time this week: it made assembly highlighting look like a job
that needed doing twice, and its isinstance branches in the follow and xrefs
handlers were unreachable twins of the ListingView branch directly below them,
which is exactly the kind of thing you read carefully before realising it can't
execute.
Gone with it: the dead branches (folded into the ListingView ones, keeping the
fall-through-edge comment that was worth keeping), its CSS rule, and the
comments that pointed at it as though it were a live alternative.
DisasmModel STAYS — the domain still uses it to index a function's instructions
(_do_edit_item resolves a row within a function that way). Only the widget was
dead.
209/0 scenarios, 26/0 blob, 30/0 project UI, 36/0 index, 32/0 formats, 39/0
project, 27/0 pool. Smoke-tested a live pane afterwards: the listing renders,
highlighted, and `drive where` answers.
|
|
The listing showed the mnemonic bright and every operand in one body colour.
IDA already classifies each token, for every processor it supports:
generate_disasm_line() emits \x01<tag>text\x02<tag> and the tag says what the
text IS. We were calling tag_remove() and throwing that away.
So: no lexer. A pygments asm lexer would be a worse guess and would need one
dialect per architecture — this is arch-correct for free, including the ARM/MIPS
blobs the loader work just made openable.
lea rcx, function; "usage"
insn reg punct name cmt
_idatui_spans() parses the tags into [[kind, text], ...], heads rows carry
"spans", Head.spans holds them, and _span_segments() renders them with a
fallback to the old mnemonic/rest split for older workers.
Palette rule: NEUTRALS for the machine (mnemonic brightest — it's the column you
scan; registers at body weight because they're most of the text), HUES only where
they mean something (numbers, strings, symbols), structure recedes so commas and
brackets stop competing with operands.
Two things that fail SILENTLY and are now encoded:
* The constants are SCOLOR_DATNAME / SCOLOR_CODNAME. There is no SCOLOR_DNAME —
a wrong guess leaves the tag unmapped, symbols render as plain body text, and
nothing tells you why. Probed the live IDA to get the real names.
* Spans must be whitespace-collapsed exactly as `text` is, walking characters
rather than per span, because a run of IDA's column padding straddles span
boundaries. A row only gets spans when they reconstruct `text` exactly, so a
mismatch degrades to the old rendering instead of corrupting the line.
The reason this was parked yesterday was NOT a bug in it. listing_view's
"undefining a data head yields an unknown run" waits for
`index_of_ea(dea) >= 0` — but dea is the head it just undefined, so it is in the
OLD model too and the predicate passes instantly, asserting against pre-edit
rows. It only ever passed because the model swap won the race; spans made pages
3x bigger, the swap lost, and the check accused working code. It now waits for
the model to be REPLACED.
Cost measured on libcrypto: 95KB per 500-row page, 50ms; model ensure(2000)
228ms. Acceptable for what it buys.
tests: new asm_highlight scenario (+7) — >90% of code rows carry spans, insn/reg/
punct present, every span kind has a style, spans reconstruct the row text
exactly, mnemonic is the first span. 202/0 scenarios, 26/0 blob, 30/0 project UI.
TODO: DisasmView appears to be dead code (never instantiated; Ctx.dis returns
ListingView), which is why this only needed doing once.
|
|
bad answer
Three bugs, reported together, with one shared root: you couldn't get to the
address field, so the address went into the processor filter, so IDA got a
nonsense processor name and refused to open — and the app dead-ended with a
misleading error.
**Tab never reached any modal.** Binding("tab,shift+tab", "toggle_view",
priority=True) is an APP binding, and priority bindings run before the focus
chain. Nothing in any dialog in this app could ever be tabbed to; the load
dialog is just where it finally mattered. action_toggle_view now hands the key
back when a modal is up, which fixes it everywhere.
**...and DOM order was the wrong tab order anyway.** focus_next() stopped at the
processor list, which is arrow-driven and has nothing to type. LoadOptionsScreen
overrides it to cycle the two fields you actually type into.
**...and the dialog outgrew the terminal.** With the palette's default
max-height the 21-row list pushed the address field and help line off the bottom
of the screen. Nothing errors — the field simply isn't there, which reads as
"Tab does nothing". Capped per-dialog.
**Project mode never asked.** _should_ask_load_options bailed on
`self._project is not None` with the comment "project mode carries per-binary
options already" — true only if someone had already filled them in. A raw blob
added to a project got the silent x86-at-0 treatment the dialog exists to
prevent. Now asked at boot AND on switching to an undescribed binary, and the
answer is written back to the project entry (Project.set_load), so it is asked
once per binary, not once per run.
**A rejected answer dead-ended.** Getting a processor wrong is an ordinary
mistake; it left an empty app with "connect failed: worker exited (code 1)" and
a message blaming a locked .i64. The worker now names the real suspect when load
switches were in play, and the app re-opens the dialog instead of giving up.
Verified with real keys in a tmux pane, which is the only way any of this shows
up: Tab -> address field -> 0x8000000 -> Enter -> 35 functions at 0x80039AC; a
bogus processor -> "those load options were rejected — try again" with the
dialog back; project mode -> asks, loads at the right base, and the answer is in
the project file.
tests: +3 scenarios (Tab moves focus under a modal, lands on the address field,
cycles back). 202/0 scenarios, 39/0 project, 32/0 formats, 30/0 project UI.
docs/TEXTUAL_NOTES.md gets the priority-binding and clipped-modal traps.
|
|
Last commit let you SAY how to load a blob. This one notices when you should
have. Interactive IDA pops a dialog when no loader matches; we silently loaded
as x86 at 0 and analysed to nothing, so the flag only helped people who already
knew they needed it — which is exactly the people who don't need help.
formats.sniff() recognises the formats IDA definitely handles (ELF, PE, Mach-O,
dex, wasm, ar, COFF, Intel HEX, S-records). Anything else gets
LoadOptionsScreen: a filterable processor list with human labels, a load-address
field, Enter to accept, Esc to load it the way IDA would have anyway.
Deliberate asymmetry: the sniff only claims formats it is sure about. A false
"unknown" costs one dismissible dialog; a false "known" is the silent wrong
answer this exists to kill. Esc is always an escape hatch.
The list offers 20 processors, not IDA's 73 — most of the rest are museum
pieces, and a name typed into the filter that matches nothing is taken literally
so nothing is actually unreachable. Endianness is spelled out (arm vs armb)
because getting it backwards is the most common route to zero functions.
Asked only when nobody has answered yet: not with --processor, not in project
mode (entries carry their own), and not when a database exists — the .i64
already records how the image was loaded.
Textual trap worth recording: the screen stored the file size in self._size,
which is Widget's own backing field for outer_size. Assigning an int to it
crashes layout with "'int' object has no attribute 'region'" from deep inside
_set_dirty, nowhere near the cause. Same family as the _render collision.
The paragraph conversion now lives in exactly one place (formats.load_args);
BinaryRef and launch both call it.
Verified on a real AArch64 blob: dialog appears, filtering to "arm" leaves two
entries, base 0x8000000 accepted -> "-parm -b800000" -> 35 functions at
0x8002440. An ELF never asks, and neither does a blob that already has a .i64.
tests: new tests/test_formats.py (21) and a load_options scenario asserting the
dialog stays out of the way for a recognised binary. 199/0 scenarios, 39/0
project, 30/0 project UI, 36/0 index, 27/0 pool, 21/0 formats.
|
|
The app carried three overlapping ideas of the current pane: _active, _pref, and
Textual focus. 190e28b tied focus to _active in split; this removes _pref, which
turns out never to have been a variable at all.
_pref was assigned "listing" in __init__ and "listing" on a project binary
switch. Nothing else ever wrote it. But _code_view() branched on it:
return self.query_one(DecompView if self._pref == "decomp" else ListingView)
so it always returned the listing, whatever you were reading. Its two callers put
focus back after the goto prompt closes — so cancelling `g` while in the
pseudocode focused the HIDDEN listing, and the pane you were looking at stopped
answering the keyboard. Arrows did nothing until you clicked. (It also explains
why routing follow through _code_view() earlier made Enter a dead key: the helper
had been quietly lying the whole time.)
_code_view() now returns the active code pane. _pref is gone from the app,
BinaryState and the RPC snapshot keeps "pref" for wire compat, sourced from
_code_mode() — the one place that answers "which code view do we return to from
hex", and a constant by design in the unified layout.
Verified on a live pane both ways: g then Esc in pseudocode, then two Downs.
Fixed, the cursor moves 0 -> 2; with the old lookup restored it sits at 0 with
focus=ListingView. Same check added to the view_toggle scenario, and it fails
without the fix. 195/0, project UI 23/23.
|
|
Walking back with Esc ended in a dead keypress: nav_depth dropped but the screen
didn't change. Measured on a live pane, four Escs after one follow:
Esc #1 -> sub_3500 L7 (where we followed from) good
Esc #2 -> main L54 decomp (where we followed from) good
Esc #3 -> main L344 listing good
Esc #4 -> main L344 listing -- nav 2 -> 1, nothing moved
The app auto-lands on main at startup, and opening main again from Ctrl+N
appended a second, identical entry. Every "navigate to where you already are"
did this; the extra Esc it bought is invisible except that it does nothing, which
is exactly what "back is broken" feels like from the keyboard.
Both push sites now go through _push_nav, which replaces the top entry instead of
appending when the target is the same (ea, view, line) — the newer entry's
metadata still wins. _same_spot compares dec_cursor for pseudocode and cursor for
the listing.
Note what is NOT a duplicate: the first follow after Tab-to-pseudocode still
pushes twice (nav 1 -> 3). That's deliberate — Tab leaves _cur transient and off
the stack, so the follow records the pseudocode position first, which is why Esc
#1 above returns to main L54 rather than dropping you into the listing. Verified
that subsequent follows push exactly one each.
tests: the palette scenario re-opens the function it is already on and asserts
nav depth is unchanged. Fails without the fix (nav 2 -> 3). 194/0.
|
|
Ctrl+N found nothing on libcrypto. Typing "PEM_read_bio" returned 0 of 10093
functions while the backend resolved the very same name to 0x1d6290.
_fuzzy lowercased the NAME but not the QUERY, then walked the query's characters
through the lowered name. One capital letter and the subsequence walk fails at
the first character, so the match is not merely worse — it is None, and the
palette shows nothing at all.
Invisible on the test binary because C symbols there are lowercase (main, strlen,
error) and every existing palette check typed a lowercase query. Fatal on any
library that capitalises: OpenSSL, most SDKs, Windows binaries. The paging index
was the obvious suspect and was innocent — all_loaded() had all 10093.
Verified on a live libcrypto pane: "PEM_read_bio" now returns 34 hits, exact
match ranked first.
tests: the palette scenario now types "MAIN" and expects "main". Confirmed it
fails without the fix (results=[]) and passes with it. 193/0.
|
|
Five sections stacked vertically ran ~45 rows, so F1 scrolled on any terminal
under ~50 lines. Each section is now its own bordered card (title in the border),
and the cards flow into as many columns as the width allows.
Textual CSS has no media queries, so the column count is computed in compose from
the real app width. Sizing off the WIDEST section would let one long row (Move's
"Ctrl+Home / Ctrl+End") inflate every column and cost a column that would
otherwise fit, so _columns() measures the actual layout instead: chunk the
sections, sum the per-chunk maxima, take the most columns that fit. Keys are
right-aligned per card rather than globally, so a card of short keys stays narrow.
200x50 -> 3 cols 140x44 -> 2 cols 100x36 -> 1 col (scrolls)
160x44 -> 3 cols 120x40 -> 2 cols
The scroll container stays, so a very small terminal degrades to scrolling
instead of clipping — but at any normal size nothing scrolls. The box hugs its
content (width:auto all the way down; VerticalScroll needed width:auto too or it
filled to max-width) and stays centred.
Tried a real CSS Grid first: it collapsed to zero height at narrow widths, and
grid sizing from on_mount read a stale app width. Explicit columns are
predictable. Also shortened one description that was pushing the widest card out.
help scenario now asserts the five cards exist and that the content fits without
a scrollbar. Suite 192/0.
|
|
Completes phase 2 — the index already carried strings (KIND_STRING), the palette
just didn't offer the toggle. StringsPalette now mirrors SymbolPalette: F2 flips
between this binary and the project, project rows are prefixed with their binary,
and choosing a literal in another binary switches to it and jumps.
Ranking matches the symbol side: the trigram index guarantees the match, so
ordering is earliest-match then shortest, with a (binary, addr) tiebreak — the
same tie that crashed symbol search when two binaries shared a name, avoided here
by construction. Project scope caps at 60 like symbols.
_results is now (binary, addr, text) tuples in both palettes, so _on_string_chosen
takes the same (binary, addr) choice and routes through _switch_then_goto.
test_project_ui.py: local scope stays single-binary, F2 spans >=2 binaries
(23 checks). Suite 191/0.
|
|
idatui/index.py — one on-disk index (<sidecar>/idx/project.db) over every binary
in a project, so search works for binaries whose worker isn't running.
Indexing choice, measured rather than guessed:
* SQLite FTS5 with the TRIGRAM tokenizer — stdlib, no dependency (nothing else
was installed and nothing is needed), and unlike a prefix index it matches
arbitrary substrings, which is what symbol names and string bodies need.
* 300k-entry corpus: 1.9 ms per query vs 11.8 ms for a Python scan and 28.9 ms
for plain LIKE; 0.2 ms per incremental insert.
* Size was the stated worry and turned out not to bite: bash contributes 5.9k
entries / 0.15MB of text, libcrypto.so.3 30.7k / 0.52MB. At ~5.7x the text a
20-binary project is ~12-23MB — against .i64 files already in the sidecar
(libcrypto's alone is 72MB), roughly 1% of what the project already costs. The
reason to be on disk is residency, not size.
* Trigram can't answer queries under 3 chars and returns nothing rather than
erroring, so search() falls back to LIKE — otherwise incremental typing would
look broken until the third keystroke.
Wiring: after a binary's functions load, its symbols + strings are folded into
the index (skipped when the source's size/mtime is unchanged). Ctrl+N gains a
scope toggle on F2 — not ctrl+a, which the focused Input binds to "home" so it
never reaches the palette. Project scope narrows via the index then ranks with
the existing _fuzzy, keeping the same feel; hits are prefixed with their binary,
and choosing one elsewhere switches binary and jumps to it.
Also fixes another instance of the Textual-markup trap: the palette titles ate
"[project]" as a style tag (same class of bug as the status bar), so the pal
titles are markup=False now.
tests/test_index.py: 24 stdlib checks — substring/case-insensitive matching, kind
filter, the <3 char fallback, multi-binary search, per-binary incremental
reindex, staleness, forget, persistence. Suite 191/0.
Strings (") still needs the same scope toggle; the index already carries them.
|
|
Quitting used to be silent AND inconsistent. Single-binary mode closed the worker
with save=False, so renames/types/comments were DROPPED without a word; project
mode did the opposite and saved everything silently via the pool. Neither told
you anything.
Now 'q'/ctrl+q routes through action_quit: clean databases exit immediately, and
anything unsaved raises a QuitScreen naming the affected databases with
s save & quit d discard & quit Esc cancel
Saving happens with an overlay up, because writing a large .i64 takes seconds and
doing it during teardown would look like a hang with no UI left to explain it.
_dirty_labels() covers project mode too: the active binary plus any still-resident
one that was edited. Evicted binaries were already saved on the way out, so they
can't be silently lost.
on_unmount now distinguishes an explicit choice from an unexpected teardown:
_save_on_exit is None (crash/kill -> save defensively, including single-binary
mode which previously discarded), False (user chose discard, or we already saved).
Verified end-to-end across two sessions on a temp copy: rename + 's' -> the rename
is still there on reopen; rename + 'd' -> it is not. Plus a quit_guard scenario
(clean exits immediately, dirty asks, Esc cancels).
Also hardens Ctx.open(view="decomp"): F5/Tab only decompiles from a focused code
pane and the listing may still be settling, so a swallowed Tab surfaced much later
as "pseudocode view shows: active=listing". It now retries instead of assuming the
first Tab takes. Full suite 191/0, green twice in a row.
|
|
Neither was a flake; both were test bugs that happened to be timing/binary
dependent.
1. filter: the scenario hardcoded the glob 'sub_1*', which matches NOTHING in a
binary whose code never reaches 0x1xxx — echo's functions are sub_2xxx..
sub_7xxx, so it failed deterministically, every run. Derive the glob from real
names instead (first sub_ prefix present) and assert the row count equals the
expected match count, which is stronger than the old 0 < n < total. A
self-check asserts the derived glob actually matches something, so this can't
silently rot again on another binary.
2. view_toggle: "tab switches to disassembly" asserted after a fixed pause(0.1),
but decomp -> listing runs through _toggle_to_listing, a BACKGROUND WORKER —
_active only flips once the listing model has loaded. The pause held in short
runs and lost the race in a full one, which is why this looked like collateral
from the filter failure. Wait for the state instead. Two related hardenings:
focus the decomp pane before Tab (elsewhere Tab is focus-next, silently
leaving us in the decompiler), and wait for the listing cursor to carry an ea
before the F5 check (F5 legitimately no-ops on an unaddressed row).
Full suite now 187 passed / 0 failed, the first fully green run — previously
174-182 with 2-6 "known flakes" whose count drifted with machine load.
|
|
The permanent Footer spent a screen row on a truncated, always-visible key list.
Remove it (the status line now owns the bottom row) and put the full cheatsheet
behind F1 — grouped by task (Navigate / Views / Move / Edit / Search) rather than
by widget, which is what makes it readable. Esc, F1 or q closes it; there's also
a "Keyboard shortcuts" command-palette entry.
F1 rather than '?' because '?' is already search-backwards in the code views.
Textual leaves F1 unbound (App only claims ctrl+q/ctrl+c), so it traps cleanly.
Gotcha worth recording: the helper that builds the cheatsheet was first called
_render, which collides with Widget._render — Textual invoked ours internally and
got a rich Text where it wanted a Visual, so the whole screen failed to paint
("'Text' object has no attribute 'render_strips'"). Renamed to _cheatsheet.
The search scenario's "rendered above the footer" check now asserts the search
input owns the bottom row instead. New `help` scenario: footer gone, F1 opens it,
groups + real bindings present, Esc closes (5 checks).
Suite note: view_toggle fails in a full run ONLY as collateral from the standing
`filter` flake, which runs immediately before it — when filter leaves the table
empty, view_toggle's open_biggest has nothing to open. Verified: view_toggle
passes alone (13/13) and directly after `help` (18/18), and reproduces as a
cascade with --only filter,view_toggle. Not a regression from this change.
|
|
The pseudocode view still had the old mapping (home -> goto_top, end ->
goto_bottom), so <End> jumped you to the bottom of the function instead of the
end of the line. Bring it in line with the listing view:
home start of line ctrl+home top of the function
shift+home first non-blank ctrl+end/G bottom of the function
end end of line
shift+home skips the C indentation — the pseudocode analogue of the listing's
skip-the-address-gutter.
Verified live and locked in view_toggle (5 checks): <end> stays on the line with
the scroll unchanged, <home> hits column 0, <shift+home> lands on the indent
width, and ctrl+home/ctrl+end still reach top/bottom. Full suite 179/2-flake.
|
|
In the decompiler, 'y' on a local variable already worked (func_types -> lvars ->
set_lvar_type), but a GLOBAL fell through every case and silently retyped the
ENCLOSING FUNCTION'S PROTOTYPE — worse than not working, since the prompt said
"prototype" while you thought you were typing a variable.
* server/patch_server.py: new data_type tool — {addr,name,type,size,is_func} for
a data item, so the prompt can prefill the current type and the caller can tell
a global from a function.
* domain: Program.data_type() + set_data_type() (set_type with kind="global").
* app: _prepare_retype gains the data case between "function" and the
current-function fallback, with a size-based prefill when the global is still
untyped; _do_retype routes kind="data" to set_data_type.
Classification verified on echo/main: 'v3' -> lvar (prefill 'char *'), 'stdout'
-> data (prefill 'FILE *'), 'main' -> func prototype, an unresolvable token ->
the enclosing prototype (unchanged fallback).
Also fixes a latent crash found while probing this: on_listing/decomp_view_
cursor_moved called self.query_one(ListingView), but App.query_one searches the
TOP screen — a cursor-moved message landing while any modal is up (loading
overlay, project switch) raised NoMatches out of a message handler and killed the
app. Both handlers now go through _try_view().
Pilot `retype` extended to 9 checks covering all three flavours, each asserting
the other targets are left alone. Two of the new checks needed settles: applying
a retype recompiles asynchronously, so scanning/indexing the pseudocode without
waiting reads text that's about to be replaced (this also cut the scenario from
30s to 2.6s of previously-wasted timeout). Full suite 174/2-flake.
|
|
The sync only ran off CursorMoved, so a wheel-scroll or scrollbar drag — which
moves the viewport but never the cursor — left the other pane behind.
* ListingView/DecompView post a new Scrolled message from watch_scroll_y when the
rounded scroll changes; the app re-syncs on it (guarded to the active pane, so
a companion's align() can't feed back).
* _split_anchor(view): the sync now anchors on the driver's cursor while it is
visible, else on the top visible row. Cursor moves behave exactly as before
(key-nav always scrolls the cursor into view); once a pure scroll takes the
cursor off-screen the viewport itself becomes the anchor, so the companion
keeps following what you're actually looking at — including re-decompiling as
you scroll across function boundaries.
Pilot split_view gains a pure-scroll check (viewport moves, cursor doesn't, the
companion still moves): 20/20. Full suite 167/2-flake.
Three existing checks were setting .cursor without scrolling it into view, which
the anchor correctly treats as "cursor not visible"; they now scroll like real
key-nav. The multi-region check also had to make the decomp the ACTIVE pane
before a decomp-driven sync — otherwise the companion's align() fires Scrolled
and re-syncs listing-driven, clobbering the link (impossible in real usage, where
the companion is by definition not the active pane).
|
|
Scrolling either pane left the other one wherever it happened to be: the
companion only scrolled when the linked row went off-screen (reveal()), so the
link could sit at the bottom edge while the driver's cursor was mid-viewport —
visually incoherent, your eye had to hunt for it.
Add ListingView/DecompView.align(row, screen_row): scroll so the linked row lands
at the SAME viewport offset as the driver's cursor, and use it in _sync_split for
both directions. The two panes now track each other line-for-line, so the eye
reads straight across. Best-effort at the ends (can't scroll above line 0, nor
past the end when the pseudocode is shorter than the viewport).
Pilot split_view: deterministic alignment check — park the listing cursor at a
known viewport offset deep in the function, sync, and assert the decomp's scroll
top is exactly the aligned value (accounting for both clamps). 19/19; full suite
166/2-flake.
|
|
ida-pro-mcp exposes no full strings list (only a filtered/capped "interesting"
survey), so this is a new injected tool plus a filterable browser.
* server/patch_server.py: list_strings(offset,count,min_len,refresh) — every
literal from idautils.Strings() as {addr,text,len,type}, paginated, with a
module-level cache keyed by min_len (rebuilding is O(n) and the browser pages
the whole list).
* domain: StrLit dataclass + Program.strings() — pages the full list once and
caches it.
* app: StringsPalette modal (mirrors SymbolPalette) — case-insensitive substring
filter with the match highlighted, addr/len/text columns, ↑↓/Enter/Esc.
Bodies are sanitized to one printable line (\n/\r/\t escaped, non-printables
dropped, long strings clipped) so control chars can't break the layout;
display strings are pre-rendered+pre-lowered once since filtering runs per
keystroke. Enter jumps to the literal in the unified listing via _goto_ea.
Bound to '"' and Shift+F12, plus a "Strings…" command-palette entry.
Verified on echo: 150 strings listed with addr/len/text, filtering 'usage'
narrows to 2 (case-insensitive), Enter lands the listing cursor on the literal.
Pilot `strings` scenario 6/6; full suite 165/2-flake.
|
|
The unified listing spans many functions, but the decomp pane was pinned to the
one function it was opened on — scrolling the listing cursor past that function's
bounds stopped syncing. Now _sync_split tracks the decompiled function's ea span
(_split_range, from decomp_map) and, when the listing cursor leaves it, re-points
the decomp pane to the function under the cursor: _resync_decomp (exclusive
worker) -> function_of(ea) -> _apply_resync re-decompiles + reloads the region
map, or just drops the band over data/undefined (keeping the last function).
Pilot split_view: moving the listing cursor into another function re-syncs the
decomp (18/18).
|
|
The sync direction follows the FOCUSED pane, but that was only discoverable via
Tab. Add on_descendant_focus: in split, focusing a pane (Tab or a mouse click)
makes it the leading/driver pane and re-syncs — so clicking into the pseudocode
and cursoring around now drives the listing, matching intuition (previously a
click focused the widget but left _active — and thus the sync direction —
pointing at the other pane). The split status now ends with "(Tab/click: drive
<other pane>)" so it's obvious.
Pilot split_view: clicking the pseudocode pane makes it the driver (17/17).
|
|
Polish for the split view:
* _split_status(): status line now reads "name @ ea [split · pseudocode line N
↔ K insn]" / "[split · listing]" from the focused pane, instead of the
single-view [pseudocode]/[listing] labels clobbering it. Wired into both
cursor-moved handlers and _apply_decomp.
* min-width gate: action_toggle_split refuses to enter split below
_SPLIT_MIN_WIDTH (100 cols) so two usable code panes always have room.
* navigation keeps both panes on the same function: verified (not fixed —
goto/follow in split already routes _open_entry -> _show_active split branch,
reloading the listing + decomp + region map for the new function).
Review turned two roadmap items into non-issues: split is a persistent mode
orthogonal to nav entries (back/forward just navigate within split), and search
is already per-focused-pane.
Pilot split_view gains: split-aware status, goto-in-split navigates, and both
panes reload on navigation (16/16). Full suite 157/2-flake. Split view is
feature-complete (phases 1-4).
|
