| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
| |
a terminal"
This reverts commit baf599e143dbf23f33886ba03a60c5d9f3c19078.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Esc-to-go-back needed two presses. It looked like the decomp pane was swallowing
the first one, but the RPC state dump before/after a single Esc settles it:
nav_depth 3 -> 3 action_back never ran
cursor.col 19 -> 20 +1 column, i.e. `right`
modal null nothing was up to eat the key
The keypress arrived as a right-arrow. \x1b is both Escape and the lead byte of
every arrow/function-key sequence (ESC [ C is right), so a terminal or tmux that
merges a lone Esc with what follows delivers something else entirely. Nothing in
the nav path touches cursor_x and there is no widget-level escape binding on the
code views, so this was never application logic — I spent two commits looking in
the wrong place before asking for the dump.
Bind back to "escape,backspace". Backspace is unambiguous, so going back never
depends on a byte the terminal can reinterpret; Esc still works wherever it's
delivered intact. tmux users generally want `set -sg escape-time 10` regardless.
docs/TEXTUAL_NOTES.md records the symptom and, more usefully, the triage: check
nav_depth in the state dump first — if the action never ran, it's the input
layer, not the logic.
Verified: backspace in the pseudocode pane takes nav 3 -> 2 and reloads the
previous function. Suite 192/0.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Phase 2: strings has the F2 toggle now, so drop the "still needs it" note.
Record the 60-row project cap and the reason for it, and the ranking rule —
including why the (binary, addr) tiebreak is load-bearing (a name shared by two
binaries ties on every other field and Python then compares hit objects,
raising TypeError; that shipped as a crash and was fixed).
* Phase 3: spell out precisely how it relates to the PLT/import-stub backlog
item. An export index turns "follow strcmp -> extrn strcmp:near, dead end" into
a real jump into libc WHEN that library is in the project. It doesn't retire
the item: a single-binary session, or a provider outside the project, still
lands on a stub and still wants it recognised and presented as an import rather
than a decompiler error.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
idatui/index.py — one on-disk index (<sidecar>/idx/project.db) over every binary
in a project, so search works for binaries whose worker isn't running.
Indexing choice, measured rather than guessed:
* SQLite FTS5 with the TRIGRAM tokenizer — stdlib, no dependency (nothing else
was installed and nothing is needed), and unlike a prefix index it matches
arbitrary substrings, which is what symbol names and string bodies need.
* 300k-entry corpus: 1.9 ms per query vs 11.8 ms for a Python scan and 28.9 ms
for plain LIKE; 0.2 ms per incremental insert.
* Size was the stated worry and turned out not to bite: bash contributes 5.9k
entries / 0.15MB of text, libcrypto.so.3 30.7k / 0.52MB. At ~5.7x the text a
20-binary project is ~12-23MB — against .i64 files already in the sidecar
(libcrypto's alone is 72MB), roughly 1% of what the project already costs. The
reason to be on disk is residency, not size.
* Trigram can't answer queries under 3 chars and returns nothing rather than
erroring, so search() falls back to LIKE — otherwise incremental typing would
look broken until the third keystroke.
Wiring: after a binary's functions load, its symbols + strings are folded into
the index (skipped when the source's size/mtime is unchanged). Ctrl+N gains a
scope toggle on F2 — not ctrl+a, which the focused Input binds to "home" so it
never reaches the palette. Project scope narrows via the index then ranks with
the existing _fuzzy, keeping the same feel; hits are prefixed with their binary,
and choosing one elsewhere switches binary and jumps to it.
Also fixes another instance of the Textual-markup trap: the palette titles ate
"[project]" as a style tag (same class of bug as the status bar), so the pal
titles are markup=False now.
tests/test_index.py: 24 stdlib checks — substring/case-insensitive matching, kind
filter, the <3 char fallback, multi-binary search, per-binary incremental
reindex, staleness, forget, persistence. Suite 191/0.
Strings (") still needs the same scope toggle; the index already carries them.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Wires the project model + worker pool into the app. Project mode is ADDITIVE —
without --project the app is exactly the single-binary tool it was, which is what
keeps the 167-check pilot meaningful.
* IdaTui(project=...) builds a WorkerPool and opens the project's first binary;
_open_worker_client asks the pool instead of spawning directly.
* BinaryState snapshots what a switch leaves behind (program, func_index, nav,
cur, view prefs, filter). Switching reuses the _after_reconnect shape: swap
client+program, rebuild the index, reopen the entry. A still-resident binary
restores instantly (Program + index are in memory); an evicted one gets a fresh
worker but keeps its nav history, which is just addresses.
* ProjectPalette (Ctrl+O, + a "Switch binary…" palette command): the project's
binaries with resident/analysed/pinned/active state and memory, filterable.
* launch.py --project FILE, creating the project when binaries are also given;
stages everything up front so the source tree is never written to.
Two bugs found while testing:
* _did_auto_land is app-wide, but landing is per-binary: after the first binary
landed, a cold switch never landed at all AND left the switch overlay up
forever. Reset it per switch.
* PRE-EXISTING: the status Static had Textual markup enabled, so a single-word
bracket marker parses as a style tag and is silently eaten — [listing] and
[pseudocode] have never actually rendered (only [split · listing] survived,
because the · makes it an invalid tag). Status is plain text with brackets and
symbol names, so markup=False.
tests/test_project_ui.py: end-to-end pilot on two real binaries (18 checks) —
boot, switcher contents, switch, per-binary index, both workers resident,
switch back with state intact, return-to-where-you-were, and the promise that
the source tree stays pristine while every artifact lands in the sidecar.
Full single-binary suite unchanged at 167/2 (the standing flakes).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
First slice of multi-binary projects (docs/PROJECTS.md): the on-disk model, with
no runtime wiring yet.
idatui/project.py (stdlib-only, like domain/worker):
* Project.load/create/save — an explicit JSON project file listing binaries;
paths resolve relative to it, labels default to the basename and are
disambiguated on collision (they name files).
* A sidecar dir beside the project file (<stem>.idatui.d/) holds bin/ (staged
binaries), their .i64 + scratch, and idx/ for phase 2. IDA opens the STAGED
file, so nothing lands in the source tree — today targets/ carries ~244MB of
IDA litter around ~13MB of binaries, much of it stale wedge files.
* stage() copies rather than hardlinks. A hardlink is free but makes source and
staged one inode, so an in-place rebuild (cp over the path truncates instead of
replacing) would silently swap the bytes under an analysed DB with nothing to
detect it. The unit test caught exactly that. A copy also leaves the sidecar
self-contained once the sources are gone.
* Re-staging a changed source drops its now-stale DB; sweep_scratch() clears the
unpacked working files a hard-killed worker leaves behind (never the .i64).
tests/test_project.py: 27 checks, pure stdlib (no IDA/textual/worker), <1s.
docs/PROJECTS.md: the full design — the one-worker-per-DB constraint with
measured costs (bash worker = 126MB RSS/117MB PSS; libcrypto's DB is 72MB, so
residency is budgeted by MEMORY, not a worker count), the switch between
"switching needs a live worker" and "searching doesn't (cached index)", and
phases 1-4.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The unified listing spans many functions, but the decomp pane was pinned to the
one function it was opened on — scrolling the listing cursor past that function's
bounds stopped syncing. Now _sync_split tracks the decompiled function's ea span
(_split_range, from decomp_map) and, when the listing cursor leaves it, re-points
the decomp pane to the function under the cursor: _resync_decomp (exclusive
worker) -> function_of(ea) -> _apply_resync re-decompiles + reloads the region
map, or just drops the band over data/undefined (keeping the last function).
Pilot split_view: moving the listing cursor into another function re-syncs the
decomp (18/18).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Polish for the split view:
* _split_status(): status line now reads "name @ ea [split · pseudocode line N
↔ K insn]" / "[split · listing]" from the focused pane, instead of the
single-view [pseudocode]/[listing] labels clobbering it. Wired into both
cursor-moved handlers and _apply_decomp.
* min-width gate: action_toggle_split refuses to enter split below
_SPLIT_MIN_WIDTH (100 cols) so two usable code panes always have room.
* navigation keeps both panes on the same function: verified (not fixed —
goto/follow in split already routes _open_entry -> _show_active split branch,
reloading the listing + decomp + region map for the new function).
Review turned two roadmap items into non-issues: split is a persistent mode
orthogonal to nav entries (back/forward just navigate within split), and search
is already per-focused-pane.
Pilot split_view gains: split-aware status, goto-in-split navigates, and both
panes reload on navigation (16/16). Full suite 157/2-flake. Split view is
feature-complete (phases 1-4).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The Ghidra "region band": moving the pseudocode cursor now lights up EVERY
instruction that C line owns, not just one.
* server/patch_server.py: new decomp_map tool — sweeps cfunc.get_line_item across
each pseudocode line's columns and collects the ea from each item's dstr()
('EA: desc', matching the /*ea*/ marker source so it aligns with the display
lines). Returns {addr, lines:[{ea, eas:[...]}]}. (First tried item.get_ea(),
which reports a different ea and didn't align — dstr() is the right source.)
* domain: Program.decomp_map(ea) -> per-line ea lists, cached by name-gen.
* app: _load_split_map fetches it off-thread into _split_eamap/_split_ea2line;
_sync_split bands the full instruction region for a C line (decomp drives) and
uses the exact ea->line inverse (listing drives), falling back to the single
marker until the map lands. Maps cleared on leaving split.
Pilot split_view gains: decomp_map returns/aligns with the markers, and a
multi-instruction C line bands >1 listing row (13/13). Full suite 154/2-flake.
The idalib spike ran on the pilot's own worker (the standalone worker kept
getting reaped in this sandbox).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The Ghidra sync: in split, the focused pane drives and the companion shows a
subtle band (_S_LINK) on the linked location + scrolls it into view. The
companion's cursor never moves (band + scroll only), so there's no echo/
ping-pong and no guard is needed.
* _sync_split(source): listing drives -> DecompView.line_for_ea (largest /*ea*/
marker <= cursor ea) bands the covering C line; decomp drives ->
ListingModel.ensure_ea bands the covering instruction row.
* wired into on_listing/decomp_view_cursor_moved (gated on the focused pane),
the Tab focus-switch (re-link from the new driver), _apply_decomp (link once
the pseudocode loads) and _apply_enter_split; bands cleared on leaving split.
* ListingView/DecompView gain _link_rows/_link_line + set_link/reveal, a
render_line apply_style(_S_LINK) band, and DecompView.line_for_ea.
Still single-ea per line (one instruction highlighted) — the full instruction
range is phase 3 (the decomp_map tool). Pilot split_view now covers both sync
directions + that the band actually paints (10/10); full suite 151/2-flake.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The Ghidra-style dual view, layout + toggle (no cursor sync yet — that's phase 2).
's' (and a "Split view" palette command) toggles a _split mode where the listing
(left) and pseudocode (right) show together, divided by a keyline, one focused.
Entering split loads the listing + decompiles the current function into both
panes; Tab/F5 switches the focused pane; any single-view target (hex, etc.) or 's'
again collapses back to the focused pane.
* app: _split flag; _show_active gains a split branch (both panes, load decomp,
focus active, #panes.split class); action_toggle_split + _enter_split worker;
action_toggle_view switches panes when split; 's' binding + palette entry;
#panes.split ListingView divider CSS.
* docs/SPLIT_VIEW.md: the full design + phased roadmap (the hard part — line<->EA
set mapping via a sweep of cfunc.get_line_item — is scoped for phase 3).
* tests: split_view scenario (enter/load/tab-focus/exit, 5 checks). Also fix
disasm_nav's stale goto-bottom (press ctrl+end; plain 'end' is end-of-line now).
Verified live over RPC (both panes render, Tab flips focus, 's' exits) and pilot
split_view 5/5.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* RPC.md: the TUI is launched via `./ida-tui <bin> --rpc <sock>` (was
`idatui.tui --db`); point the "regression lock" note at rpcclient/drive +
the pilot instead of the deleted rpc_smoke.py.
* PAGING_FINDINGS.md: reframe the intro as the ida-pro-mcp tool functions the
worker now calls in-process (shapes/caps unchanged); replace the
supervisor-era "idle self-exit" + "max_workers cap" sections with the
single-owned-worker lifecycle (no cap, no idle-exit, crash -> reconnect).
* TEXTUAL_NOTES.md / TUI_DRIVING_BLUEPRINT.md: drop the ida-pro-mcp framing and
the deleted rpc_smoke.py references (-> test_scenarios.py / rpcclient).
Also updated the idatui + idatui-rpc skills (in ~/.pi, outside the repo) to the
worker model: no supervisor/spawn.sh/--db, worker python + $IDATUI_WORKER_PYTHON,
worker.py/worker_client.py/errors.py architecture, and the load-starvation gotcha.
|
| |
|
|
|
|
|
|
|
|
| |
A design reference for bringing the idatui spawn/drive/introspect experience
to other terminal apps. Captures the invariants worth keeping, the reusable
core vs per-target Adapter split, a target taxonomy (embedded / control-
channel / black-box PTY), the three hard problems (injection, settle,
introspection) with technique gradients, tmux as the universal substrate,
and a concrete gdb worked example (TUI pane + gdb-Python plugin as the
adapter). No code.
|
| |
|
|
|
|
|
|
|
| |
The comment prompt is single-line, so a long note rendered as one giant
'//' line that ran off the right edge and got clipped. Interpret a literal
'\\n' (backslash-n) in the comment text as a real newline in _do_comment
(the single choke point for the ';' prompt and the RPC 'comment' verb) —
Hex-Rays renders each as its own '//' line. Verified live and locked in
the comment_func scenario (multi-line render); suite 104 green.
|
| |
|
|
|
|
|
|
| |
Round out idatui.drive with 'save' and 'retype <fn> <proto>' so the whole
common RE loop (orient/understand/act/persist) has a terse command. Update
docs/RPC.md and both skills (idatui, idatui-rpc) to recommend idatui.drive
as the day-to-day driving surface, with rpcclient/raw as the fallback for
unwrapped verbs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The clock/title Header added noise (and made screen() non-deterministic);
remove it from the app itself — the Footer stays. Layout shifts up a row;
scenario suite still 101 green.
xrefs_from on a function was near-useless: the server's xrefs_from is
address-scoped, so passing the entry ea only returned the fall-through
from the first instruction. For a function target it now returns
whole-body references from the decompiler (callees + string/data refs:
{to,name,string,is_func,type}); an explicit 0xADDR stays address-scoped.
Verified live: xrefs_from main now lists setlocale/getopt_long/sub_* etc.
rpc_smoke: 25 green.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
cursor_on {word,line?,occurrence?} places the cursor on a token, verified
with the app's own tokenizer (so 'main' won't match inside 'domain');
rename/retype/follow gain an optional word= that runs it first — the
ergonomic way to edit a named symbol without a manual view+cursor dance.
Single-driver gate: the socket now serves one client at a time; a second
concurrent connection is refused with 'busy' (no multi-driver support yet,
by request). Sequential connections are unaffected.
screen gains format=text|html|svg (html/svg are colored — for an
out-of-band web viewer). rpc_smoke: 24 green.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Introspection (offloaded to a thread so a big fetch can't freeze the
render): pseudocode (full body), disassembly (bounded), xrefs_to/from
(structured), resolve (name->ea). These let an agent reason about a
function and the call graph without scraping screen().
Semantic: select (choose the highlighted/nth item in the open xrefs or
symbol-palette list and activate it — the natural follow-up to xrefs/
symbols), search (/ or ? incremental in the active code view), save
(Ctrl+S). rpc_smoke exercises all of them end-to-end (18 green).
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Replace the docked names pane as the primary way to jump to symbols with a
command-palette overlay: Ctrl+N opens SymbolPalette, type to fuzzy-find (scored
subsequence match with matched-char highlighting), up/down to select, Enter to
open, Esc to close. Mouse-clickable too.
The docked FunctionsPanel now starts hidden and is still reachable/toggleable
with Ctrl+B (classic table view, sort, filter, goto all unchanged); a code view
takes focus on startup. Status hints 'Ctrl+N: find symbol'.
Pilot checks for open/fuzzy/subsequence/select/close. Tests reveal the docked
pane (Ctrl+B) for the existing table-driven checks. full suite 74/74.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
DecompView.goto derived its scroll with _scroll_cursor_into_view() -> a plain
scroll_to, which updates scroll_offset but doesn't repaint at the new offset
(Textual only repaints on a rounded scroll change, and there's no layout pass
since the pane isn't reloaded). So a same-function xref jump landed the cursor
correctly but painted the old scroll until the next cursor move nudged a repaint.
Route goto's scroll (derived or explicit) through _apply_scroll, which re-applies
after the next refresh with layout=True -- the same pattern show() uses. The
cross-function path already worked because reloading + clearing the loading cover
forces a fresh paint. Document the gotcha in TEXTUAL_NOTES.
full suite 64/64.
|
| |
|
|
|
|
|
| |
Companion to PAGING_FINDINGS.md capturing the hard-won Textual behaviour
(bindings/mixins, ScrollView repaint & scroll-clamp, loading cover blurs focus,
mouse offset mapping, Input/Footer overlap, no cpp grammar) and the app patterns
(name-gen cache invalidation, cursor+scroll history, name+address follow).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Root cause of 'Maximum idalib worker count reached': the app bumped idle_ttl to
~never on every open, so sessions never freed and hit the default cap of 4.
- app: drop the immortal bump_idle_ttl(1e9); rely on the KeepAlive heartbeat
(120s) to keep the running session warm, and open with a moderate idle_ttl
(1800s) so the worker self-exits and frees its slot after the TUI closes.
- spawn.sh: set IDA_MCP_MAX_WORKERS (default 8) for headroom.
- doc: supervisor prunes unreachable workers before counting, so killing an idle
worker frees a slot on the next open.
(Verified separately: big/truncated decompile results propagate renames fine via
the earlier force_recompile fix.)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Root cause: idalib workers run an idle watchdog (worker_lifecycle.py) that
self-exits after idle_ttl_sec (default 600s) of no requests -- deliberate
resource hygiene for a shared/ephemeral supervisor, wrong for an interactive
TUI. CLI-opened startup binary always gets 600s (no flag).
Fix (both verified against a 12s-TTL session):
- IDAClient.bump_idle_ttl(1e9): re-open (idempotent) re-applies TTL, no upper
cap -> effectively immortal. Primary fix, call once at startup.
- IDAClient.keepalive()/KeepAlive: background server_health heartbeat resets the
watchdog; covers adopted sessions too. Safety net.
tests/test_keepalive.py: 4/4 (heartbeat + bump both keep a short-TTL worker
alive past 2x TTL). docs updated with why + fix.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
- Program: model registry + small prefetch pool; cache invalidation hooks
- FunctionIndex: lazy pagination (advance by len, clamp page=500), filter globs,
viewport windows; full 10,092-func enumerate matches survey in ~3.1s
- DisasmModel: block-cached windowed disasm (256/block), forward+back prefetch,
cached instruction totals; deep window revisit 196ms -> 0ms (cache proven)
- Decompilation: truncation-aware, hard-failure surfaced as data (monster funcs)
- resolve(): int/hex/symbol via lookup_funcs (string-array query shape)
- tests/test_domain.py: 17/17 on both ls (~290 funcs) and libcrypto (10k+52k-insn)
- smoke_client no longer assumes 'main' exists (works on libraries)
- doc: idle_ttl worker self-exit finding ("not reachable" needs re-open)
|
|
|
Measured against libcrypto.so.3 (10,092 funcs, biggest 52,120 insns):
- list_* count cap ~700, disasm max_instructions cap ~500, then SILENT
collapse to 10 (not clamped) -> must clamp client-side (use 500).
- pagination must advance by len(data); next_offset=offset+count skips data.
- disasm offset paging is O(offset): 6ms@0 -> 180ms@50k, no resumable cursor
-> domain layer must cache windows + prefetch + over-fetch.
- include_total scans whole func (~217ms on monster) -> fetch once, cache.
- decompile hard-fails on huge funcs as a soft error (code=null) -> handle.
- idb_open needs a writable path for the .i64.
tests/stress_paging.py: durable paging benchmark harness
docs/PAGING_FINDINGS.md: constraints that drive the paging layer
|
