aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorblasty <blasty@local>2026-07-25 23:03:10 +0200
committerblasty <blasty@local>2026-07-25 23:03:10 +0200
commitb8d9aa4fce78f4523cffaba8b46095d2dfb66310 (patch)
tree97e73ec6b027a19cbeafe026aaf8b90bad0773d1
parentprojects phase 3: follow an import into the binary that implements it (diff)
downloadida-tui-b8d9aa4fce78f4523cffaba8b46095d2dfb66310.tar.gz
ida-tui-b8d9aa4fce78f4523cffaba8b46095d2dfb66310.tar.xz
ida-tui-b8d9aa4fce78f4523cffaba8b46095d2dfb66310.zip
projects phase 4: cross-binary back, linkage-guided pre-warm, project verbs
Three things, all following from phase 3 making cross-binary jumps ordinary. **A cross-binary jump was a one-way door.** Nav history is per-binary, so arriving in another binary — a project search hit, or now following an import into the library that implements it — landed you in an empty history with nothing to take you back. _switch_then_goto records the binary it came FROM, and action_back falls through to that hop once local history is spent: Esc walks back through the function you were in, then the binary you were in. Manual Ctrl+O switching records nothing, because that isn't navigation. **Pre-warm follows the linkage graph, not list order.** _prewarm_provider warms the binary providing the most of this one's imports — where a follow is most likely to go, so its startup is paid before you ask for it. "Next in the list" would have been arbitrary; phase 3 gave us something better to ask. pool.prewarm() refuses rather than making room. Evicting a binary the user visited to speculatively load one they haven't is a straight downgrade, and it throws away that binary's caches as well; at a tight budget pre-warm just does nothing. The cost of a worker that doesn't exist yet can only be estimated, so it uses the largest resident one (same program, different database) — and if that estimate proves wrong, the speculative worker is the one evicted, never a chosen one. **Driving a project.** pane spawn --project FILE [--open BIN]; `binaries` lists the inventory (active / resident / indexed / where Esc returns to) and `switch {binary,addr?}` makes another active — with an address it takes the search-hit path, so it records a hop. state gains `binary` and `hops`, which it should have had the moment project mode existed. Verified on real sessions: drive binaries/switch against an echo+cat project pane; Esc crossing back from a switch; and prewarm on echo+libc picking libc (provider of echo's imports) and warming it after an evict. tests: +5 pool (prewarm warms, no-ops when resident, refuses at budget, evicts nothing when refusing, ignores unknown labels) and +4 project UI (jump records the hop, Esc crosses back, hop consumed). Confirmed the Esc-back checks fail with the branch removed. 195/0 scenarios, 27/0 project UI, 36/0 index, 27/0 pool, 33/0 project. Left open: project-level persistence across sessions.
-rw-r--r--docs/PROJECTS.md33
-rw-r--r--idatui/app.py52
-rw-r--r--idatui/drive.py26
-rw-r--r--idatui/pane.py32
-rw-r--r--idatui/pool.py31
-rw-r--r--idatui/rpc.py40
-rw-r--r--tests/test_pool.py29
-rw-r--r--tests/test_project_ui.py33
8 files changed, 264 insertions, 12 deletions
diff --git a/docs/PROJECTS.md b/docs/PROJECTS.md
index e0484ad..e5359e0 100644
--- a/docs/PROJECTS.md
+++ b/docs/PROJECTS.md
@@ -197,9 +197,36 @@ original fix — recognise the thunk and present it as an import ("strcmp —
imported, provider not in project") rather than surfacing a decompiler error.
Phase 3 covers the valuable half; stub *presentation* stays worth doing.
-**Phase 4 — polish.** Background pre-warm, nav-history policy (per-binary to
-start; a global stack is a later question), RPC/`drive` verbs (`binaries`,
-`switch`), project-level persistence.
+**Phase 4 — polish (mostly done).**
+
+*Nav history is per-binary, with a cross-binary way back.* Each binary keeps its
+own stack in `BinaryState`; switching restores it (addresses outlive the worker,
+so it survives eviction). But phase 3 made a cross-binary jump an ordinary
+action, and arriving in a binary whose history is empty made it a one-way door.
+`_switch_then_goto` now records the binary it came FROM, and `action_back` falls
+through to that hop once local history is spent. So Esc walks back through the
+function you were in, then the binary you were in. Manual switching (Ctrl+O)
+records nothing — that's not navigation.
+
+*Pre-warm follows the linkage graph, not list order.* When a binary finishes
+indexing, `_prewarm_provider` warms the binary that provides the most of its
+imports — where a follow is most likely to take you, so its startup is paid
+before you ask. `WorkerPool.prewarm()` refuses rather than evicting: spending a
+binary you visited on one you haven't is a straight downgrade, and it would throw
+away that binary's caches too. At a tight budget pre-warm simply does nothing. It
+estimates the cost of a not-yet-spawned worker from the largest resident one,
+which is the only evidence available; if the estimate proves wrong, the
+speculative worker is the one that goes.
+
+*Driving a project.* `pane spawn --project FILE [--open BIN]` opens a project
+pane. `binaries` lists the inventory (active / resident / indexed count / where
+Esc returns to) and `switch {binary,addr?}` makes another one active — with an
+address it goes through the same path as a search hit, so it records a hop. The
+state snapshot now carries `binary` and `hops`.
+
+Still open: project-level persistence (remember the active binary and each
+binary's position across sessions — today a fresh launch auto-lands).
+
## Decisions
diff --git a/idatui/app.py b/idatui/app.py
index 39a4b94..cef58f1 100644
--- a/idatui/app.py
+++ b/idatui/app.py
@@ -3170,6 +3170,7 @@ class IdaTui(App):
self._states: dict[str, BinaryState] = {}
self._pending_restore = None # entry to reopen after a switch
self._goto_after_switch = None # cross-binary search hit to land on
+ self._hops: list[str] = [] # binaries a navigation crossed FROM
self._nav_seq = 0 # bumped per navigation; drops stale ones
# None = teardown wasn't an explicit quit (crash/kill): save defensively.
# False = the user chose discard, or we already saved on the way out.
@@ -3486,6 +3487,41 @@ class IdaTui(App):
self.app.call_from_thread(self._auto_land)
self._index_binary() # project mode: keep the cross-binary index fresh
+ @work(thread=True, exclusive=True, group="prewarm")
+ def _prewarm_provider(self) -> None:
+ """Warm the binary this one leans on hardest, once we're idle.
+
+ Not "the next in the list" — phase 3 tells us something better. The
+ binary providing the most of this one's imports is where a follow is
+ most likely to take you, so paying its startup now is the switch you
+ would otherwise wait for. Refuses to evict anything (see pool.prewarm),
+ so at a tight budget this simply does nothing.
+ """
+ if self._pool is None or self._index is None or self._binary is None:
+ return
+ try:
+ imps, _ = self.program.linkage()
+ except Exception: # noqa: BLE001
+ return
+ if not imps:
+ return
+ from collections import Counter
+ votes: Counter = Counter()
+ for name in {i.name for i in imps}:
+ for h in self._index.providers(name, exclude=self._binary):
+ votes[h.binary] += 1
+ resident = set(self._pool.resident())
+ cand = next((b for b, _ in votes.most_common() if b not in resident), None)
+ if cand is None:
+ return
+ n = votes[cand]
+ try:
+ if self._pool.prewarm(cand):
+ self.app.call_from_thread(
+ self._status, f"pre-warmed {cand} (provides {n} imports)")
+ except Exception: # noqa: BLE001 -- speculative work must never surface
+ pass
+
@work(thread=True, exclusive=True, group="index")
def _index_binary(self) -> None:
"""Fold this binary's symbols + strings into the project index, so it can
@@ -3516,6 +3552,7 @@ class IdaTui(App):
return
self.app.call_from_thread(
self._status, f"indexed {self._binary}: {n} symbols, strings + linkage")
+ self._prewarm_provider()
# -- initial landing --------------------------------------------------- #
#: function names tried (in order) as the startup landing spot
@@ -3695,7 +3732,15 @@ class IdaTui(App):
def _switch_then_goto(self, binary: str, addr: int) -> None:
"""A project-wide hit in another binary: switch to it, then jump there
- once its index has loaded."""
+ once its index has loaded.
+
+ Records the hop so Esc can come back. Nav history is per-binary, so
+ without this a cross-binary jump — a project search hit, or following an
+ import into the library that implements it — is a one-way door: you
+ arrive in a binary whose history is empty and nothing takes you back.
+ """
+ if self._binary is not None and binary != self._binary:
+ self._hops.append(self._binary)
self._goto_after_switch = addr
self._switch_binary(binary)
@@ -4117,6 +4162,11 @@ class IdaTui(App):
dest = self._nav[-1]
self._status(f"\u25c2 back to {dest.name}\u2026")
self._open_entry(dest, push=False)
+ elif self._hops:
+ # Local history is spent, but we got here from another binary.
+ label = self._hops.pop()
+ self._status(f"\u25c2 back to {label}\u2026")
+ self._switch_binary(label) # _states restores its nav and position
elif self.query_one("#left", FunctionsPanel).display:
table.focus()
else:
diff --git a/idatui/drive.py b/idatui/drive.py
index 6c96335..b6fa641 100644
--- a/idatui/drive.py
+++ b/idatui/drive.py
@@ -165,6 +165,31 @@ def cmd_names(c, args):
or "(no match)"
+def cmd_binaries(c, args):
+ """Project inventory: which binaries, which is active, which have a live
+ worker, how much of each is indexed."""
+ r = c.call("binaries")
+ out = []
+ for b in r["binaries"]:
+ mark = "*" if b["active"] else ("~" if b["resident"] else " ")
+ out.append(f" {mark} {b['label']:<28} indexed={b['indexed']:<7} {b['source']}")
+ if r.get("hops"):
+ out.append(f" (Esc returns to: {' <- '.join(r['hops'])})")
+ return "\n".join(out) + "\n * active ~ worker resident"
+
+
+def cmd_switch(c, args):
+ if not args:
+ raise SystemExit("usage: switch <binary> [addr|name]")
+ p = {"binary": args[0]}
+ if len(args) > 1:
+ a = args[1]
+ p["addr"] = a if a.lower().startswith("0x") else c.call("resolve", name=a)["ea"]
+ r = c.call("switch", **p)
+ fn = (r.get("function") or {}).get("name")
+ return f" now on {r.get('binary') or args[0]}" + (f" @ {fn}" if fn else "")
+
+
def _rename_one(c, old, new):
c.call("goto", target=old, delay_ms=0)
st = c.call("rename", name=new, word=old, delay_ms=0)
@@ -232,6 +257,7 @@ COMMANDS = {
"callees": cmd_callees, "callers": cmd_callers, "names": cmd_names,
"rename": cmd_rename, "mv": cmd_mv, "note": cmd_note, "retype": cmd_retype,
"save": cmd_save, "screen": cmd_screen, "raw": cmd_raw,
+ "binaries": cmd_binaries, "switch": cmd_switch,
}
diff --git a/idatui/pane.py b/idatui/pane.py
index e34b822..53afef3 100644
--- a/idatui/pane.py
+++ b/idatui/pane.py
@@ -131,15 +131,20 @@ def spawn(args) -> int:
if not os.environ.get("TMUX"):
print("error: not inside tmux (spawn creates a tmux pane)", file=sys.stderr)
return 2
- if not args.open:
- print("error: pass --open <binary>", file=sys.stderr)
+ if not args.open and not getattr(args, "project", None):
+ print("error: pass --open <binary> or --project <file>", file=sys.stderr)
return 2
sock = args.sock or os.path.join(_sockdir(), f"idatui-{secrets.token_hex(3)}.sock")
- target = os.path.abspath(os.path.expanduser(args.open))
- if not os.path.exists(target):
+ project = (os.path.abspath(os.path.expanduser(args.project))
+ if getattr(args, "project", None) else None)
+ target = os.path.abspath(os.path.expanduser(args.open)) if args.open else None
+ if target is not None and not os.path.exists(target):
print(f"error: no such binary: {target}", file=sys.stderr)
return 2
+ if project is not None and target is None and not os.path.exists(project):
+ print(f"error: no such project: {project}", file=sys.stderr)
+ return 2
# Reap workers leaked by previously-stopped/crashed panes so we don't spawn
# into a full IDA_MCP_MAX_WORKERS (which makes the new TUI hang forever,
@@ -151,7 +156,15 @@ def spawn(args) -> int:
# the command the pane runs: the launcher spawns a private idalib worker for
# this binary and becomes the TUI, so kill-pane tears the whole thing down.
- inner = [args.python, "-m", "idatui.launch", target, "--rpc", sock]
+ if project is not None:
+ # launch takes: --project FILE [binaries...]; extra binaries are added to
+ # the project (and a missing project file is created from them).
+ inner = [args.python, "-m", "idatui.launch", "--project", project]
+ if target is not None:
+ inner.append(target)
+ inner += ["--rpc", sock]
+ else:
+ inner = [args.python, "-m", "idatui.launch", target, "--rpc", sock]
cmd = f"cd {REPO!r} && exec " + " ".join(_q(a) for a in inner)
split = ["split-window", "-v" if args.vertical else "-h",
@@ -166,8 +179,8 @@ def spawn(args) -> int:
split.append(cmd)
pane = _tmux(*split)
- row = {"sock": sock, "pane": pane, "target": target,
- "kind": "open", "started": time.time()}
+ row = {"sock": sock, "pane": pane, "target": project or target,
+ "kind": "project" if project else "open", "started": time.time()}
reg = [r for r in _load_registry() if r.get("sock") != sock]
reg.append(row)
_save_registry(reg)
@@ -301,8 +314,11 @@ def main(argv: list[str]) -> int:
sub = p.add_subparsers(dest="cmd", required=True)
sp = sub.add_parser("spawn", help="open a TUI pane and wait until ready")
- sp.add_argument("--open", metavar="PATH", required=True,
+ sp.add_argument("--open", metavar="PATH",
help="binary to open (its dir must be writable)")
+ sp.add_argument("--project", metavar="FILE",
+ help="project file to open instead of a single binary; "
+ "any --open paths are added to it (created if absent)")
sp.add_argument("--sock", help="RPC socket path (default: auto in $XDG_RUNTIME_DIR)")
sp.add_argument("--python", default=DEFAULT_PY, help=f"python for the TUI ({DEFAULT_PY})")
sp.add_argument("--vertical", action="store_true", help="split vertically (stacked)")
diff --git a/idatui/pool.py b/idatui/pool.py
index df852f8..25ea879 100644
--- a/idatui/pool.py
+++ b/idatui/pool.py
@@ -129,6 +129,37 @@ class WorkerPool:
self._enforce_budget(protect=label)
return client
+ def prewarm(self, label: str, progress=None) -> bool:
+ """Spawn a worker for ``label`` only if it fits the budget AS IT STANDS.
+
+ Pre-warming must never cost residency: evicting a binary the user
+ actually visited to speculatively load one they haven't is a straight
+ downgrade, and the eviction would also throw away that binary's caches.
+ So this refuses rather than making room, and returns False.
+
+ The cost of a worker that doesn't exist yet can only be estimated; the
+ largest resident one is the best evidence available (they are all the
+ same program with a different database). With nothing resident we have
+ no evidence at all, so we allow one — that is the case where the budget
+ is certainly free.
+ """
+ if label in self._clients:
+ return False
+ if self.project.by_label(label) is None:
+ return False
+ used = self.memory_mb()
+ est = max((self._mem(c) for c in self._clients.values()), default=0)
+ if used + est > self.budget_mb:
+ return False
+ self.get(label, progress=progress)
+ # get() enforces the budget protecting the NEW label; if that had to
+ # evict, our estimate was wrong and the speculative worker is the one
+ # that should go — never a binary the user chose.
+ if self.memory_mb() > self.budget_mb and label != self.active:
+ self.evict(label)
+ return False
+ return True
+
def _touch(self, label: str) -> None:
if label in self._lru:
self._lru.remove(label)
diff --git a/idatui/rpc.py b/idatui/rpc.py
index dd0c9c6..0921b89 100644
--- a/idatui/rpc.py
+++ b/idatui/rpc.py
@@ -72,6 +72,8 @@ METHODS = {
"search": "{term,direction?=1} incremental search in the code view",
"select": "{index?} choose the highlighted/nth item in the open modal",
"save": "persist the .i64 (Ctrl+S)",
+ "binaries": "-> project binaries {label,active,resident,indexed} (project mode)",
+ "switch": "{binary,addr?} make another project binary active (addr also jumps)",
"close": "dismiss a modal (Escape)",
"move": "{dir,n?=1} fast movement (down/up/.../pagedown)",
"cursor": "{line?,col?} set the code-pane cursor directly",
@@ -173,7 +175,9 @@ def snapshot(app) -> dict[str, Any]:
"cursor": _cursor_info(app, w),
"status": st,
"filter": app._filter_term,
+ "binary": app._binary, # None outside project mode
"nav_depth": len(app._nav),
+ "hops": list(getattr(app, "_hops", [])),
"dirty": bool(app._dirty),
"modal": _modal_snapshot(app),
**_readiness(app),
@@ -526,6 +530,42 @@ class RpcServer:
if method == "functions":
return functions(app, params.get("filter"), int(params.get("limit", 50)))
+ # -- projects ------------------------------------------------------ #
+ if method == "binaries":
+ if app._project is None:
+ raise ValueError("not a project session (launch with --project)")
+ counts = app._index.counts() if app._index is not None else {}
+ resident = set(app._pool.resident()) if app._pool is not None else set()
+ return {"active": app._binary, "hops": list(app._hops),
+ "binaries": [{"label": r.label, "source": r.source,
+ "active": r.label == app._binary,
+ "resident": r.label in resident,
+ "indexed": int(counts.get(r.label, 0))}
+ for r in app._project.refs]}
+
+ if method == "switch":
+ if app._project is None:
+ raise ValueError("not a project session (launch with --project)")
+ label = str(params.get("binary") or params.get("label") or "")
+ if app._project.by_label(label) is None:
+ have = ", ".join(r.label for r in app._project.refs)
+ raise ValueError(f"no such binary {label!r} (have: {have})")
+ addr = params.get("addr")
+ if label == app._binary and addr is None:
+ return snapshot(app)
+ if addr is None:
+ app._switch_binary(label)
+ else:
+ # Same path a project search hit takes, so it records a hop and
+ # Esc comes back here.
+ app._switch_then_goto(label, int(str(addr), 0)
+ if isinstance(addr, str) else int(addr))
+ await settle(app, lambda: app._binary == label
+ and app._func_index is not None
+ and app._func_index.complete,
+ timeout=float(params.get("timeout", 300.0)))
+ return snapshot(app)
+
# -- structured introspection (heavy: run off the UI loop) -------- #
loop = asyncio.get_running_loop()
if method == "pseudocode":
diff --git a/tests/test_pool.py b/tests/test_pool.py
index d4347d9..5c6e2c4 100644
--- a/tests/test_pool.py
+++ b/tests/test_pool.py
@@ -155,6 +155,35 @@ def main() -> int:
check("default budget is derived, not a fixed worker count",
pool3.budget_mb >= 256, pool3.budget_mb)
+ # -- prewarm: speculative, and never at the cost of a real binary ------ #
+ with tempfile.TemporaryDirectory() as tmp:
+ proj = _mkproject(tmp, n=3)
+ made2 = {}
+
+ def spawn2(ref, ttl):
+ c = FakeClient(ref, mem=100)
+ made2[ref.label] = c
+ return c
+
+ pool = WorkerPool(proj, budget_mb=250, spawn=spawn2,
+ mem_fn=lambda c: c.mem)
+ labels = [r.label for r in proj.refs]
+ a, b, c_ = labels[0], labels[1], labels[2]
+ pool.get(a)
+ pool.set_active(a)
+ check("prewarm warms a binary when the budget has room",
+ pool.prewarm(b) is True and b in pool.resident(), f"{pool.resident()}")
+ check("prewarm is a no-op for something already resident",
+ pool.prewarm(b) is False)
+ # 2 x 100MB resident, estimate 100 more -> 300 > 250: must refuse
+ check("prewarm refuses rather than making room",
+ pool.prewarm(c_) is False and c_ not in pool.resident(),
+ f"resident={pool.resident()} mem={pool.memory_mb()}/{pool.budget_mb}")
+ check("refusing to prewarm evicts nothing",
+ set(pool.resident()) == {a, b}, f"{pool.resident()}")
+ check("prewarm ignores a label outside the project",
+ pool.prewarm("nope") is False)
+
print(f"\n{PASS} passed, {FAIL} failed")
return 1 if FAIL else 0
diff --git a/tests/test_project_ui.py b/tests/test_project_ui.py
index 731811d..47e511a 100644
--- a/tests/test_project_ui.py
+++ b/tests/test_project_ui.py
@@ -168,6 +168,39 @@ async def run(bins):
await pilot.press("escape")
await pilot.pause(0.2)
+ # -- a cross-binary jump is not a one-way door ----------------- #
+ # Nav history is per-binary, so arriving in another binary lands you
+ # in an empty history. Esc must fall through to the binary you came
+ # from, or a project search hit (and, since phase 3, following an
+ # import) strands you.
+ here, there = app._binary, (first if app._binary == second else second)
+ hops0 = len(app._hops)
+ target = app._index.search("main", limit=200)
+ tgt = next((h for h in target if h.binary == there), None)
+ if tgt is None:
+ check("cross-binary jump records a hop", False, "no hit in the other binary")
+ else:
+ app._switch_then_goto(tgt.binary, tgt.addr)
+ jumped = await settle(lambda: app._binary == there
+ and app._func_index is not None
+ and app._func_index.complete, 180)
+ check("a project hit switches to the other binary", jumped,
+ f"binary={app._binary} want={there}")
+ check("the jump records where it came from",
+ len(app._hops) == hops0 + 1 and app._hops[-1] == here,
+ f"hops={app._hops}")
+ # spend the local history first, then Esc must cross back
+ for _ in range(6):
+ if not app._hops or app._binary != there:
+ break
+ await pilot.press("escape")
+ await pilot.pause(0.6)
+ returned = await settle(lambda: app._binary == here, 180)
+ check("Esc crosses back to the binary the jump came from",
+ returned, f"binary={app._binary} want={here} hops={app._hops}")
+ check("the hop is consumed, not repeated",
+ not app._hops, f"hops={app._hops}")
+
# -- and the same toggle for strings --------------------------- #
from idatui.app import StringsPalette
await pilot.press("quotation_mark")