diff options
| author | blasty <blasty@local> | 2026-07-25 22:33:16 +0200 |
|---|---|---|
| committer | blasty <blasty@local> | 2026-07-25 22:33:16 +0200 |
| commit | dcb76f92e8851f747e17cae848e800a394a073e1 (patch) | |
| tree | 7b2ef9e92049e7a394f707e313cb79ba65309ced | |
| parent | nav: one notion of "which pane you're in" — delete _pref (diff) | |
| download | ida-tui-dcb76f92e8851f747e17cae848e800a394a073e1.tar.gz ida-tui-dcb76f92e8851f747e17cae848e800a394a073e1.tar.xz ida-tui-dcb76f92e8851f747e17cae848e800a394a073e1.zip | |
projects phase 3: follow an import into the binary that implements it
Following a call to strcmp reached the PLT/extern entry and stopped there —
Hex-Rays has nothing to decompile, because the code lives in a library this
binary only references. With the other binary open in the same project we
already had everything needed to cross that gap; we just weren't indexing it.
Index each binary's imports and exports (KIND_IMPORT / KIND_EXPORT) alongside its
functions and strings. On a follow, _import_stub asks whether the target address
is one of this binary's import stubs; if so _cross_binary_impl asks the index who
exports that name, and we switch there instead of landing on the thunk.
Verified end to end on a real echo + libc project: Enter on `strrchr(a1, 47)` in
echo's pseudocode switches to libc.so.6 and lands on strrchr at 0xaf960.
Three things it turns on:
* ELF symbol versioning. The importer sees strrchr@@GLIBC_2.2.5 while the
provider may export any of three spellings, so raw names resolve almost
nothing. domain.link_name() cuts at the first '@'; Linkage.raw keeps what IDA
reported, which is what the listing shows.
* Exact match, not substring — ProjectIndex.exact(), so `read` doesn't bind to
pread/read_line/thread_start. It also answers below the 3-char trigram floor,
and plenty of real exports are that short.
* Resolution reads the on-disk index, so a provider resolves while its worker is
evicted. That's what the index was for.
When nothing in the project provides the symbol _follow_import declines and the
normal navigation runs: landing on the stub is still the honest answer, and a
single-binary session is unchanged. The PLT-stub PRESENTATION item stays open —
an unprovided import should say "imported, provider not in project" rather than
show a decompiler error.
server/patch_server.py gains list_linkage (idautils.Entries + enum_import_names);
a worker without it degrades to no linkage rather than failing.
tests: index join +8 (exact vs substring, short names, exclude-self, reverse
join, kind isolation, forget unresolves) and link_name +4. 36/0 index, 195/0
scenarios, 23/0 project UI, 33/0 project, 22/0 pool.
| -rw-r--r-- | docs/PROJECTS.md | 51 | ||||
| -rw-r--r-- | idatui/app.py | 59 | ||||
| -rw-r--r-- | idatui/domain.py | 58 | ||||
| -rw-r--r-- | idatui/index.py | 34 | ||||
| -rw-r--r-- | server/patch_server.py | 35 | ||||
| -rw-r--r-- | tests/test_index.py | 71 |
6 files changed, 290 insertions, 18 deletions
diff --git a/docs/PROJECTS.md b/docs/PROJECTS.md index cb240ac..e0484ad 100644 --- a/docs/PROJECTS.md +++ b/docs/PROJECTS.md @@ -158,23 +158,44 @@ is load-bearing: without it, a name shared by two binaries (`main`, `textdomain` most of libc) ties on every other field and Python falls through to comparing the hit objects, which raises `TypeError`. -**Phase 3 — cross-binary linking.** Index what each binary *imports* and -*exports*, then join them: "who in the project calls this export", and following -an import in A lands on the real implementation in B. +**Phase 3 — cross-binary linking (done).** Each binary's imports and exports go +into the project index alongside its functions and strings (`KIND_IMPORT`, +`KIND_EXPORT`), and following an import resolves it to the binary that provides +it. -This is where the standing **PLT/import-stub** backlog item mostly goes away. -Today, following a call to `strcmp` reaches the PLT thunk and stops at -`extrn strcmp:near` — Hex-Rays has nothing to decompile because the code lives in -a library the app never opened. With an export index, that stub resolves to -`libc`'s implementation *when that library is in the project*, so the dead end -becomes a normal cross-binary jump. +Following a call to `strcmp` used to reach the PLT/extern entry and stop: +Hex-Rays has nothing to decompile because the code lives in a library this +binary only references. Now `_follow_import` checks whether the target address is +one of this binary's import stubs, asks the index who exports that name, and +switches there. From an `echo` + `libc` project, Enter on `strrchr(a1, 47)` lands +on `strrchr` at `0xaf960` in libc. -It does not retire the item completely: a single-binary session, or an import -whose provider isn't in the project, still lands on a stub, and that case still -wants the original fix — recognise the thunk and present it as an import -("strcmp — imported, provider not in project") instead of surfacing a decompiler -error. So phase 3 covers the valuable half; stub *presentation* stays worth doing -on its own. +Three things this depends on: + +*ELF symbol versioning.* The importer sees `strrchr@@GLIBC_2.2.5`; the provider +may export `strrchr`, `strrchr@GLIBC_2.2.5`, or the versioned spelling. Comparing +raw names resolves almost nothing, so `domain.link_name()` cuts at the first `@` +and both sides meet on the bare symbol. `Linkage.raw` keeps what IDA reported, +which is what the listing shows. + +*Exact match, not substring.* The join uses `ProjectIndex.exact()`, not +`search()`: an import must bind to its own name, or `read` picks up `pread`, +`read_line` and `thread_start`. Exact match also works below the 3-character +trigram floor, which matters — plenty of real exports are one or two characters. + +*Resolution reads the index, not a worker.* A provider resolves while its worker +is evicted; that is the reason the index is on disk. The switch itself then pages +the provider in through the normal pool path. + +When nothing in the project provides the symbol, `_follow_import` declines and +the local navigation proceeds — landing on the stub is still the honest answer, +and a single-binary session behaves exactly as before. + +This does not retire the **PLT/import-stub** backlog item. An import whose +provider isn't in the project still lands on a stub, and that case wants the +original fix — recognise the thunk and present it as an import ("strcmp — +imported, provider not in project") rather than surfacing a decompiler error. +Phase 3 covers the valuable half; stub *presentation* stays worth doing. **Phase 4 — polish.** Background pre-warm, nav-history policy (per-binary to start; a global stack is a later question), RPC/`drive` verbs (`binaries`, diff --git a/idatui/app.py b/idatui/app.py index ff383be..39a4b94 100644 --- a/idatui/app.py +++ b/idatui/app.py @@ -3495,7 +3495,7 @@ class IdaTui(App): ref = self._project.by_label(self._binary) if ref is None or not self._index.is_stale(self._binary, ref.source): return - from .index import KIND_FUNC, KIND_STRING + from .index import KIND_EXPORT, KIND_FUNC, KIND_IMPORT, KIND_STRING idx = self._func_index entries = [(KIND_FUNC, f.addr, f.name) for f in (idx.all_loaded() if idx else [])] try: @@ -3504,12 +3504,18 @@ class IdaTui(App): except Exception: # noqa: BLE001 -- symbols alone are still worth indexing pass try: + imps, exps = self.program.linkage() + entries += [(KIND_IMPORT, i.addr, i.name) for i in imps] + entries += [(KIND_EXPORT, e.addr, e.name) for e in exps] + except Exception: # noqa: BLE001 -- an old worker has no list_linkage + pass + try: n = self._index.reindex(self._binary, entries, source=ref.source) except Exception as e: # noqa: BLE001 self.app.call_from_thread(self._status, f"indexing failed: {e}") return self.app.call_from_thread( - self._status, f"indexed {self._binary}: {n} symbols + strings") + self._status, f"indexed {self._binary}: {n} symbols, strings + linkage") # -- initial landing --------------------------------------------------- # #: function names tried (in order) as the startup landing spot @@ -4228,8 +4234,55 @@ class IdaTui(App): if tgt is None or tgt.to is None: self.app.call_from_thread(self._status, "nothing to follow here") return + if self._follow_import(tgt.to): + return self._do_navigate(tgt.to, push=True) + def _import_stub(self, ea: int) -> str | None: + """The import name at ``ea``, if ``ea`` is one of this binary's import + stubs. That's the dead end phase 3 exists to open up: a call to strcmp + reaches the PLT/extern entry and Hex-Rays has nothing to decompile, + because the code lives in a library this binary only references.""" + try: + imps, _ = self.program.linkage() + except Exception: # noqa: BLE001 + return None + for i in imps: + if i.addr == ea: + return i.name + return None + + def _cross_binary_impl(self, name: str) -> tuple[str, int] | None: + """``(binary, addr)`` of a project binary that EXPORTS ``name``. + + Reads the on-disk index, so a provider resolves even when its worker was + evicted — the whole reason the index exists. + """ + if self._index is None or self._project is None or not name: + return None + try: + hits = self._index.providers(name, exclude=self._binary) + except Exception: # noqa: BLE001 + return None + return (hits[0].binary, hits[0].addr) if hits else None + + def _follow_import(self, ea: int) -> bool: + """Follow an import stub into the binary that implements it. True when + it was handled (caller must not also navigate locally).""" + name = self._import_stub(ea) + if not name: + return False + found = self._cross_binary_impl(name) + if found is None: + # Leave the local navigation alone: landing on the stub is still the + # honest answer when nothing in the project provides the symbol. + return False + label, addr = found + self.app.call_from_thread( + self._status, f"{name} \u2192 {label} (import resolved)") + self.app.call_from_thread(self._switch_then_goto, label, addr) + return True + @work(thread=True, group="nav") def _follow_decomp(self, line: str, word: str | None, line_ea: int | None = None) -> None: @@ -4257,6 +4310,8 @@ class IdaTui(App): if addr is None: self.app.call_from_thread(self._status, "nothing to follow on this line") return + if self._follow_import(addr): + return # Following FROM pseudocode: keep the reader in the decompiler when the # target is decompilable, instead of dropping to the linear listing. self._do_navigate(addr, push=True, prefer_decomp=True) diff --git a/idatui/domain.py b/idatui/domain.py index ef37ab9..681542e 100644 --- a/idatui/domain.py +++ b/idatui/domain.py @@ -176,6 +176,35 @@ class StrLit: type: str = "" +def link_name(raw: str) -> str: + """A linkage name reduced to what actually joins across binaries. + + ELF symbol versioning means the importer sees ``strrchr@@GLIBC_2.2.5`` while + the provider may export ``strrchr``, ``strrchr@GLIBC_2.2.5`` or the versioned + spelling — comparing raw names silently resolves almost nothing. Cut at the + first '@' so both sides meet on the bare symbol. + """ + n = (raw or "").strip() + at = n.find("@") + return n[:at] if at > 0 else n + + +@dataclass(frozen=True) +class Linkage: + """One import or export: a name this binary takes from, or offers to, other + modules. ``module`` is set for imports (the library IDA attributes it to), + ``ordinal`` for exports. + + ``name`` is the joinable name; ``raw`` keeps the spelling IDA reported, which + is what the user sees in the listing. + """ + addr: int + name: str + module: str = "" + ordinal: int = 0 + raw: str = "" + + @dataclass class Decompilation: ea: int @@ -818,6 +847,7 @@ class Program: self._decomp: dict[int, tuple[Decompilation, int]] = {} self._decomp_maps: dict[int, tuple[list[list[int]], int]] = {} # line->ea sets self._strings: list["StrLit"] | None = None # whole-binary string literals + self._linkage: tuple[list["Linkage"], list["Linkage"]] | None = None self._name_gen = 0 # bumped on rename; invalidates stale name caches self._segments_cache: list[tuple[int, int, int, str]] | None = None self._sections: list[tuple[int, int, str]] | None = None @@ -1312,6 +1342,34 @@ class Program: self._strings = out return out + def linkage(self) -> tuple[list[Linkage], list[Linkage]]: + """``(imports, exports)`` for this binary, cached. ``([], [])`` if the + tool is unavailable — an old worker must not break the caller.""" + with self._lock: + hit = self._linkage + if hit is not None: + return hit + try: + payload = self.client.call("list_linkage", kind="both") + except IDAToolError: + return ([], []) + if not isinstance(payload, dict): + return ([], []) + imps = [Linkage(addr=_as_int(r.get("addr", 0)), + name=link_name(r.get("name", "")), + module=r.get("module", "") or "", + raw=r.get("name", "") or "") + for r in payload.get("imports", []) if isinstance(r, dict)] + exps = [Linkage(addr=_as_int(r.get("addr", 0)), + name=link_name(r.get("name", "")), + ordinal=int(r.get("ordinal", 0) or 0), + raw=r.get("name", "") or "") + for r in payload.get("exports", []) if isinstance(r, dict)] + out = ([i for i in imps if i.name], [e for e in exps if e.name]) + with self._lock: + self._linkage = out + return out + def decomp_map(self, ea: int) -> list[list[int]]: """Per-pseudocode-line instruction coverage for the split-view region highlight: a list aligned to the decompiled lines, each the EAs the diff --git a/idatui/index.py b/idatui/index.py index e7e4a41..751ad1f 100644 --- a/idatui/index.py +++ b/idatui/index.py @@ -32,6 +32,9 @@ MIN_TRIGRAM = 3 KIND_FUNC = "func" KIND_STRING = "string" +#: Cross-binary linkage: what a binary takes from, and offers to, other modules. +KIND_IMPORT = "import" +KIND_EXPORT = "export" @dataclass(frozen=True) @@ -152,6 +155,37 @@ class ProjectIndex: return [] # malformed FTS expression: treat as no matches return [Hit(binary=b, kind=k, addr=int(a), text=t) for b, k, a, t in rows] + def exact(self, name: str, kind: str, exclude: str | None = None) -> list[Hit]: + """Every entry whose text is EXACTLY ``name``, for the linkage join. + + Deliberately not ``search()``: an import must resolve to the export of + that name, not to everything containing it (``read`` would otherwise + match ``pread``, ``read_line``, ``thread_start``). Exact match also + works below the trigram floor, which matters — plenty of real exports + are one or two characters. + """ + n = (name or "").strip() + if not n: + return [] + sql = "SELECT binary, kind, addr, text FROM entries WHERE text = ? AND kind = ?" + args: list = [n, kind] + if exclude: + sql += " AND binary <> ?" + args.append(exclude) + rows = self._db.execute(sql + " ORDER BY binary, addr", args).fetchall() + return [Hit(binary=b, kind=k, addr=int(a), text=t) for b, k, a, t in rows] + + def providers(self, name: str, exclude: str | None = None) -> list[Hit]: + """Binaries in the project that EXPORT ``name`` (skip ``exclude``, the + binary asking). This is the 'follow an import to its implementation' + half of the join.""" + return self.exact(name, KIND_EXPORT, exclude) + + def importers(self, name: str, exclude: str | None = None) -> list[Hit]: + """Binaries in the project that IMPORT ``name`` — 'who in the project + calls this export'.""" + return self.exact(name, KIND_IMPORT, exclude) + # -- introspection ------------------------------------------------------ # def counts(self) -> dict[str, int]: """Indexed entry count per binary.""" diff --git a/server/patch_server.py b/server/patch_server.py index 09bca86..0e43ea5 100644 --- a/server/patch_server.py +++ b/server/patch_server.py @@ -733,6 +733,41 @@ def list_strings( "total": len(items), "next_offset": offset + len(page), } + +@tool +@idasync +def list_linkage( + kind: Annotated[str, "'import', 'export' or 'both'"] = "both", +) -> dict: + """What this binary imports from, and exports to, other modules: + {imports:[{addr,name,module}], exports:[{addr,name,ordinal}]}. Feeds the + project-wide import/export join, which resolves a PLT stub in one binary to + the real implementation in another.""" + import idaapi + import idautils + import ida_nalt + want = str(kind or "both").lower() + imports = [] + exports = [] + if want in ("import", "both"): + n = ida_nalt.get_import_module_qty() + for i in range(n): + mod = ida_nalt.get_import_module_name(i) or "" + + def _cb(ea, name, ordinal, _mod=mod): + # An ordinal-only import has no name; skip rather than invent one. + if name: + imports.append({"addr": hex(ea), "name": name, "module": _mod}) + return True + + ida_nalt.enum_import_names(i, _cb) + if want in ("export", "both"): + for index, ordinal, ea, name in idautils.Entries(): + if name: + exports.append({"addr": hex(ea), "name": name, + "ordinal": int(ordinal)}) + return {"imports": imports, "exports": exports, + "n_imports": len(imports), "n_exports": len(exports)} ''' SNIPPET = f"{BEGIN}\n{BODY.strip()}\n{END}\n" diff --git a/tests/test_index.py b/tests/test_index.py index d1e8a48..e14f715 100644 --- a/tests/test_index.py +++ b/tests/test_index.py @@ -11,7 +11,8 @@ import tempfile import time sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) -from idatui.index import KIND_FUNC, KIND_STRING, ProjectIndex # noqa: E402 +from idatui.index import (KIND_EXPORT, KIND_FUNC, KIND_IMPORT, # noqa: E402 + KIND_STRING, ProjectIndex) PASS = FAIL = 0 @@ -116,6 +117,74 @@ def main() -> int: idx.counts() == {"libfoo": 1} and idx.search("socket bind") == [], f"{idx.counts()}") + # -- cross-binary linkage join (phase 3) ------------------------------ # + idx.reindex("app", [ + (KIND_FUNC, 0x1000, "main"), + (KIND_IMPORT, 0x2000, "strcmp"), + (KIND_IMPORT, 0x2008, "read"), + (KIND_IMPORT, 0x2010, "SSL_new"), + ]) + idx.reindex("libc", [ + (KIND_EXPORT, 0x8000, "strcmp"), + (KIND_EXPORT, 0x8100, "read"), + (KIND_EXPORT, 0x8200, "pread"), + (KIND_EXPORT, 0x8300, "read_line"), + (KIND_FUNC, 0x8000, "strcmp"), + ]) + idx.reindex("libssl", [(KIND_EXPORT, 0x9000, "SSL_new")]) + + prov = idx.providers("strcmp", exclude="app") + check("an import resolves to the binary that exports it", + [(h.binary, h.addr) for h in prov] == [("libc", 0x8000)], + f"{[(h.binary, hex(h.addr)) for h in prov]}") + + # The whole point of exact(): substring search would drag in pread, + # read_line and thread_start, and 'read' is also below the trigram floor + # for some engines — an import must bind to its exact name or nothing. + prov = idx.providers("read", exclude="app") + check("the join is exact, not substring", + [(h.binary, h.addr) for h in prov] == [("libc", 0x8100)], + f"{[(h.binary, h.text) for h in prov]}") + + check("a short name still resolves (below the trigram floor)", + [h.binary for h in idx.providers("SSL_new", exclude="app")] == ["libssl"], + f"{idx.providers('SSL_new')}") + + check("an unprovided import resolves to nothing", + idx.providers("dlopen", exclude="app") == []) + + check("exclude keeps a binary from resolving to itself", + idx.providers("strcmp", exclude="libc") == [], + f"{idx.providers('strcmp', exclude='libc')}") + + imp = idx.importers("strcmp") + check("the reverse join finds who imports an export", + [(h.binary, h.addr) for h in imp] == [("app", 0x2000)], + f"{[(h.binary, hex(h.addr)) for h in imp]}") + + check("kind keeps functions out of the linkage join", + [h.binary for h in idx.providers("strcmp")] == ["libc"], + "a KIND_FUNC row named strcmp must not answer as an export") + + idx.forget("libc") + check("forgetting a provider unresolves its imports", + idx.providers("strcmp", exclude="app") == []) + + # -- ELF symbol versioning -------------------------------------------- # + # The importer sees strrchr@@GLIBC_2.2.5 while the provider may export a + # different spelling; raw names would resolve almost nothing. link_name + # cuts at the first '@' so both sides meet on the bare symbol. + from idatui.domain import link_name + check("link_name strips an ELF version suffix", + link_name("strrchr@@GLIBC_2.2.5") == "strrchr", + link_name("strrchr@@GLIBC_2.2.5")) + check("link_name leaves an unversioned name alone", + link_name("strrchr") == "strrchr") + check("link_name handles a single-@ version", + link_name("SSL_new@OPENSSL_3.0.0") == "SSL_new") + check("link_name doesn't eat a leading @", + link_name("@weird") == "@weird", link_name("@weird")) + # -- persistence ------------------------------------------------------ # path = idx.path idx.close() |
