diff options
| author | blasty <blasty@local> | 2026-07-25 14:33:05 +0200 |
|---|---|---|
| committer | blasty <blasty@local> | 2026-07-25 14:33:05 +0200 |
| commit | 27fd6fb26c16e1c8fc6e21ebb116aa9127b72520 (patch) | |
| tree | 0cb3689b2c570d7c92edfd6061e7526720ff2a45 | |
| parent | exit: ask before quitting with unsaved database changes (diff) | |
| download | ida-tui-27fd6fb26c16e1c8fc6e21ebb116aa9127b72520.tar.gz ida-tui-27fd6fb26c16e1c8fc6e21ebb116aa9127b72520.tar.xz ida-tui-27fd6fb26c16e1c8fc6e21ebb116aa9127b72520.zip | |
projects: project-wide symbol search over a SQLite FTS5 index (phase 2)
idatui/index.py — one on-disk index (<sidecar>/idx/project.db) over every binary
in a project, so search works for binaries whose worker isn't running.
Indexing choice, measured rather than guessed:
* SQLite FTS5 with the TRIGRAM tokenizer — stdlib, no dependency (nothing else
was installed and nothing is needed), and unlike a prefix index it matches
arbitrary substrings, which is what symbol names and string bodies need.
* 300k-entry corpus: 1.9 ms per query vs 11.8 ms for a Python scan and 28.9 ms
for plain LIKE; 0.2 ms per incremental insert.
* Size was the stated worry and turned out not to bite: bash contributes 5.9k
entries / 0.15MB of text, libcrypto.so.3 30.7k / 0.52MB. At ~5.7x the text a
20-binary project is ~12-23MB — against .i64 files already in the sidecar
(libcrypto's alone is 72MB), roughly 1% of what the project already costs. The
reason to be on disk is residency, not size.
* Trigram can't answer queries under 3 chars and returns nothing rather than
erroring, so search() falls back to LIKE — otherwise incremental typing would
look broken until the third keystroke.
Wiring: after a binary's functions load, its symbols + strings are folded into
the index (skipped when the source's size/mtime is unchanged). Ctrl+N gains a
scope toggle on F2 — not ctrl+a, which the focused Input binds to "home" so it
never reaches the palette. Project scope narrows via the index then ranks with
the existing _fuzzy, keeping the same feel; hits are prefixed with their binary,
and choosing one elsewhere switches binary and jumps to it.
Also fixes another instance of the Textual-markup trap: the palette titles ate
"[project]" as a style tag (same class of bug as the status bar), so the pal
titles are markup=False now.
tests/test_index.py: 24 stdlib checks — substring/case-insensitive matching, kind
filter, the <3 char fallback, multi-binary search, per-binary incremental
reindex, staleness, forget, persistence. Suite 191/0.
Strings (") still needs the same scope toggle; the index already carries them.
| -rw-r--r-- | docs/PROJECTS.md | 31 | ||||
| -rw-r--r-- | idatui/app.py | 130 | ||||
| -rw-r--r-- | idatui/index.py | 172 | ||||
| -rw-r--r-- | tests/test_index.py | 133 | ||||
| -rw-r--r-- | tests/test_scenarios.py | 16 |
5 files changed, 450 insertions, 32 deletions
diff --git a/docs/PROJECTS.md b/docs/PROJECTS.md index 1f39c05..8af1dd9 100644 --- a/docs/PROJECTS.md +++ b/docs/PROJECTS.md @@ -122,9 +122,34 @@ index, reopen the entry. A binary whose worker is still resident restores instantly (its `Program` and index are still in memory); an evicted one comes back with a fresh worker but keeps its nav history, since that is just addresses. -**Phase 2 — index cache + project-wide search.** Per-binary index (functions, -strings) persisted after first open, keyed by source size+mtime. Scope toggle in -the symbol/strings palettes, working for never-opened binaries. +**Phase 2 — index cache + project-wide search. (symbols done)** +`idatui/index.py` keeps one **SQLite FTS5 trigram** index at +`<sidecar>/idx/project.db`, populated per binary after its functions load and +re-done only when the source's size/mtime changes. + +Why that and not a library: nothing needed installing (FTS5 + the trigram +tokenizer are stdlib), and trigram indexes arbitrary *substrings*, which is what +symbol names and string bodies need. Measured on 300k entries: **1.9 ms** per +query vs 11.8 ms for a Python scan and 28.9 ms for plain `LIKE`; 0.2 ms per +incremental insert. + +Size turned out to be a non-issue. Real binaries: `bash` = 5.9k entries / +0.15 MB of text, `libcrypto.so.3` = 30.7k / 0.52 MB. The index runs ~5.7x the +text, so a 20-binary project lands around 12-23 MB — next to the `.i64` files +already in the sidecar (libcrypto's alone is 72 MB) that is ~1% of what the +project already costs. The reason to keep it on disk is **residency, not size**: +search has to work for binaries whose worker isn't running. + +Querying is two-stage: the trigram index narrows across every binary, then the +existing `_fuzzy` ranks what's left, so project scope keeps the same +fuzzy-subsequence feel as local scope. Queries shorter than 3 characters fall +back to `LIKE` — a trigram index silently returns *nothing* below that, which +would make incremental typing look broken until the third keystroke. + +`Ctrl+N` gains a scope toggle on **F2** (not `ctrl+a`: the focused `Input` binds +that to `home`). Project-scope hits are prefixed with their binary, and choosing +one in another binary switches to it and jumps. **Strings (`"`) still needs the +same toggle** — the index already carries them. **Phase 3 — cross-binary linking.** Import/export index; "who in the project calls this export"; follow an import stub in A into its implementation in B. diff --git a/idatui/app.py b/idatui/app.py index 9e415ce..bbb34a7 100644 --- a/idatui/app.py +++ b/idatui/app.py @@ -2064,17 +2064,24 @@ class SymbolPalette(ModalScreen): Binding("escape", "close", "Close"), Binding("down,ctrl+n", "cursor_down", show=False), Binding("up,ctrl+p", "cursor_up", show=False), + # F2, not ctrl+a: the focused Input binds "home,ctrl+a" so it would never + # reach us. Function keys are untouched by Input. + Binding("f2", "scope", "This binary / whole project", show=False), ] LIMIT = 200 - def __init__(self, funcs: list[Func]) -> None: + def __init__(self, funcs: list[Func], index=None, binary=None) -> None: super().__init__() self._funcs = funcs - self._results: list[Func] = [] + self._index = index # ProjectIndex, when this is a project + self._binary = binary # label of the binary we're currently in + self._project_scope = False + #: (binary|None, addr, name) — binary is None for a local hit + self._results: list[tuple] = [] def compose(self) -> ComposeResult: with Vertical(id="pal-box"): - yield Static(" symbols", id="pal-title") + yield Static(" symbols", id="pal-title", markup=False) yield Input(placeholder="fuzzy find symbol… ↑↓ select · Enter open · Esc close", id="pal-input") yield OptionList(id="pal-list") @@ -2091,35 +2098,61 @@ class SymbolPalette(ModalScreen): event.stop() self.action_choose() + def action_scope(self) -> None: + if self._index is None: + return # not a project: nothing else to search + self._project_scope = not self._project_scope + self._apply(self.query_one("#pal-input", Input).value.strip()) + def _apply(self, query: str) -> None: - if query: + # rows: (binary|None, addr, name, match positions) + if self._project_scope and self._index is not None: + from .index import KIND_FUNC + # Two stages: the trigram index narrows across every binary (cheap, + # and works for binaries with no live worker), then the same fuzzy + # ranking as local scope orders what's left. + hits = self._index.search(query, kind=KIND_FUNC, + limit=self.LIMIT * 10) if query else [] + scored = [] + for h in hits: + m = _fuzzy(h.text, query) + scored.append(((m[0] if m else 0.0), (m[1] if m else ()), h)) + scored.sort(key=lambda t: (-t[0], t[2].text)) + rows = [(h.binary, h.addr, h.text, pos) for _, pos, h in scored[:self.LIMIT]] + elif query: scored = [] for f in self._funcs: m = _fuzzy(f.name, query) if m is not None: scored.append((m[0], m[1], f)) scored.sort(key=lambda t: (-t[0], t[2].name)) - rows = [(f, pos) for _, pos, f in scored[:self.LIMIT]] + rows = [(None, f.addr, f.name, pos) for _, pos, f in scored[:self.LIMIT]] else: - rows = [(f, ()) for f in self._funcs[:self.LIMIT]] - self._results = [f for f, _ in rows] + rows = [(None, f.addr, f.name, ()) for f in self._funcs[:self.LIMIT]] + self._results = [(b, a, n) for b, a, n, _ in rows] ol = self.query_one(OptionList) ol.clear_options() opts = [] - for f, pos in rows: + for binary, addr, name, pos in rows: label = Text() - label.append(f"{f.addr:08X} ", _S_ADDR) - nm = Text(f.name) + if binary: + label.append(f"{binary:<14.14} ", _S_LABEL) + label.append(f"{addr:08X} ", _S_ADDR) + nm = Text(name) for p in pos: - if p < len(f.name): + if p < len(name): nm.stylize(_S_NAME_MATCH, p, p + 1) label.append_text(nm) opts.append(Option(label)) ol.add_options(opts) if self._results: ol.highlighted = 0 + scope = "project" if self._project_scope else "this binary" + more = "+" if len(self._results) == self.LIMIT else "" + hint = " (F2: this binary)" if self._project_scope else ( + " (F2: whole project)" if self._index is not None else "") self.query_one("#pal-title", Static).update( - f" symbols: {len(self._results)}" + ("+" if len(self._results) == self.LIMIT else "")) + f" symbols [{scope}]: {len(self._results)}{more}{hint}") def action_cursor_down(self) -> None: ol = self.query_one(OptionList) @@ -2135,11 +2168,13 @@ class SymbolPalette(ModalScreen): ol = self.query_one(OptionList) i = ol.highlighted if i is not None and 0 <= i < len(self._results): - self.dismiss(self._results[i].addr) + b, a, _ = self._results[i] + self.dismiss((b, a)) def on_option_list_option_selected(self, event: OptionList.OptionSelected) -> None: if 0 <= event.option_index < len(self._results): - self.dismiss(self._results[event.option_index].addr) + b, a, _ = self._results[event.option_index] + self.dismiss((b, a)) def action_close(self) -> None: self.dismiss(None) @@ -2175,7 +2210,7 @@ class StringsPalette(ModalScreen): def compose(self) -> ComposeResult: with Vertical(id="pal-box"): - yield Static(" strings", id="pal-title") + yield Static(" strings", id="pal-title", markup=False) yield Input(placeholder="filter strings\u2026 \u2191\u2193 select \u00b7 " "Enter jump \u00b7 Esc close", id="pal-input") yield OptionList(id="pal-list") @@ -2383,7 +2418,7 @@ class ProjectPalette(ModalScreen): def compose(self) -> ComposeResult: with Vertical(id="pal-box"): - yield Static(" binaries", id="pal-title") + yield Static(" binaries", id="pal-title", markup=False) yield Input(placeholder="filter binaries\u2026 \u2191\u2193 select \u00b7 " "Enter switch \u00b7 Esc close", id="pal-input") yield OptionList(id="pal-list") @@ -2992,12 +3027,17 @@ class IdaTui(App): self._binary: str | None = None # active project binary (label) self._states: dict[str, BinaryState] = {} self._pending_restore = None # entry to reopen after a switch + self._goto_after_switch = None # cross-binary search hit to land on # None = teardown wasn't an explicit quit (crash/kill): save defensively. # False = the user chose discard, or we already saved on the way out. self._save_on_exit: bool | None = None + self._index = None # project-wide symbol/string index if project is not None: + from .index import ProjectIndex from .pool import WorkerPool self._pool = WorkerPool(project, ttl=ttl) + self._index = ProjectIndex( + os.path.join(project.index_dir, "project.db")) self._binary = project.refs[0].label open_path = project.refs[0].staged self._open_path = open_path @@ -3297,6 +3337,32 @@ class IdaTui(App): # Land somewhere useful instead of an empty pane: main() if present, # otherwise pop the fuzzy symbol picker. self.app.call_from_thread(self._auto_land) + self._index_binary() # project mode: keep the cross-binary index fresh + + @work(thread=True, exclusive=True, group="index") + def _index_binary(self) -> None: + """Fold this binary's symbols + strings into the project index, so it can + be searched later even when its worker is gone.""" + if self._index is None or self._project is None or self._binary is None: + return + ref = self._project.by_label(self._binary) + if ref is None or not self._index.is_stale(self._binary, ref.source): + return + from .index import KIND_FUNC, KIND_STRING + idx = self._func_index + entries = [(KIND_FUNC, f.addr, f.name) for f in (idx.all_loaded() if idx else [])] + try: + entries += [(KIND_STRING, s.addr, s.text) + for s in self.program.strings()] + except Exception: # noqa: BLE001 -- symbols alone are still worth indexing + pass + try: + n = self._index.reindex(self._binary, entries, source=ref.source) + except Exception as e: # noqa: BLE001 + self.app.call_from_thread(self._status, f"indexing failed: {e}") + return + self.app.call_from_thread( + self._status, f"indexed {self._binary}: {n} symbols + strings") # -- initial landing --------------------------------------------------- # #: function names tried (in order) as the startup landing spot @@ -3307,6 +3373,13 @@ class IdaTui(App): main() if it exists; else open the symbol picker so you're never staring at a blank pane. Runs once (guarded by ``_cur``) and never steals focus from a user who already navigated.""" + goto = self._goto_after_switch + if goto is not None: # arrived here from a project-wide search hit + self._goto_after_switch = None + self._did_auto_land = True + self._dismiss_loading() + self._goto_ea(goto, push=True) + return entry = self._pending_restore if entry is not None: # switched back to a binary we'd already explored self._pending_restore = None @@ -3447,20 +3520,32 @@ class IdaTui(App): self.push_screen(StructEditor(self.program)) def action_symbols(self) -> None: - """Ctrl+N: fuzzy-find a symbol in a command-palette overlay.""" + """Ctrl+N: fuzzy-find a symbol (Ctrl+A widens it to the whole project).""" idx = self._func_index funcs = idx.all_loaded() if idx is not None else [] if not funcs: self._status("functions still loading…") return - self.push_screen(SymbolPalette(funcs), self._on_symbol_chosen) + self.push_screen(SymbolPalette(funcs, index=self._index, + binary=self._binary), + self._on_symbol_chosen) - def _on_symbol_chosen(self, addr: int | None) -> None: - if addr is None: + def _on_symbol_chosen(self, choice) -> None: # type: ignore[no-untyped-def] + if choice is None: + return + binary, addr = choice + if binary and binary != self._binary: + self._switch_then_goto(binary, addr) return f = self._func_index.by_addr(addr) if self._func_index else None self._open_function(addr, f.name if f else hex(addr)) + def _switch_then_goto(self, binary: str, addr: int) -> None: + """A project-wide hit in another binary: switch to it, then jump there + once its index has loaded.""" + self._goto_after_switch = addr + self._switch_binary(binary) + # -- projects: switching between the binaries of one target ------------ # # -- exit ---------------------------------------------------------------- # def _dirty_labels(self) -> list[str]: @@ -3588,7 +3673,10 @@ class IdaTui(App): self._func_index = st.func_index self._apply_filter(self._filter_term) # repopulate the names table self._dismiss_loading() - if self._cur is not None: + if self._goto_after_switch is not None: + addr, self._goto_after_switch = self._goto_after_switch, None + self._goto_ea(addr, push=True) + elif self._cur is not None: self._open_entry(self._cur, push=False) else: self._did_auto_land = False diff --git a/idatui/index.py b/idatui/index.py new file mode 100644 index 0000000..e7e4a41 --- /dev/null +++ b/idatui/index.py @@ -0,0 +1,172 @@ +"""ProjectIndex — one searchable index over every binary in a project. + +Phase 2 of docs/PROJECTS.md: searching across binaries must work for binaries +whose worker isn't running, so the index lives on disk rather than in the +Programs' caches. + +SQLite FTS5 with the **trigram** tokenizer, which is stdlib (no dependency) and +indexes arbitrary *substrings* rather than just word prefixes — the right shape +for symbol names and string literals. Measured on a 300k-entry corpus: 1.9 ms per +substring query (vs 11.8 ms for a Python scan and 28.9 ms for plain LIKE), 0.2 ms +per incremental insert. + +Sizing, from real binaries: libcrypto.so.3 (5.9 MB, ~10k functions + ~20k +strings) contributes 0.52 MB of text, and the index runs ~5.7x the text it +covers. A 20-binary project therefore lands around 12-23 MB — against the ``.i64`` +files already in the sidecar, where libcrypto's alone is 72 MB. The index is +roughly 1% of what the project already costs on disk. + +Caveat baked into ``search``: a trigram index cannot answer queries shorter than +three characters — it silently returns nothing rather than erroring — so short +queries fall back to LIKE. Without that, typing "e" then "er" would show "no +matches" until the third keystroke. +""" +from __future__ import annotations + +import os +import sqlite3 +from dataclasses import dataclass + +#: Trigram indexes can't match fewer than 3 characters; below this we scan. +MIN_TRIGRAM = 3 + +KIND_FUNC = "func" +KIND_STRING = "string" + + +@dataclass(frozen=True) +class Hit: + """One index match.""" + + binary: str + kind: str + addr: int + text: str + + +def _fts_phrase(query: str) -> str: + """``query`` as an FTS5 phrase: quoted so operators are literal, with any + embedded quote doubled.""" + return '"' + query.replace('"', '""') + '"' + + +class ProjectIndex: + """Symbol/string index for a whole project, keyed by binary label.""" + + def __init__(self, path: str) -> None: + self.path = os.path.abspath(path) + parent = os.path.dirname(self.path) + if parent: + os.makedirs(parent, exist_ok=True) + # check_same_thread=False: the TUI indexes from a worker thread and + # queries from the UI thread. Writes are serialised by the caller. + self._db = sqlite3.connect(self.path, check_same_thread=False) + self._db.executescript( + """ + CREATE VIRTUAL TABLE IF NOT EXISTS entries USING fts5( + text, + binary UNINDEXED, kind UNINDEXED, addr UNINDEXED, + tokenize='trigram'); + CREATE TABLE IF NOT EXISTS stamps( + binary TEXT PRIMARY KEY, + size INTEGER, mtime INTEGER, n INTEGER); + """ + ) + self._db.commit() + + # -- freshness --------------------------------------------------------- # + def stamp(self, label: str) -> tuple[int, int, int] | None: + """(size, mtime, entry count) recorded when ``label`` was last indexed.""" + row = self._db.execute( + "SELECT size, mtime, n FROM stamps WHERE binary = ?", (label,)).fetchone() + return tuple(row) if row else None # type: ignore[return-value] + + def is_stale(self, label: str, source: str) -> bool: + """True when ``label`` has never been indexed, or its source changed.""" + st = self.stamp(label) + if st is None: + return True + try: + s = os.stat(source) + except OSError: + return False # source gone: keep what we have rather than wipe it + return (st[0], st[1]) != (s.st_size, int(s.st_mtime)) + + # -- population -------------------------------------------------------- # + def reindex(self, label: str, entries, source: str | None = None) -> int: + """Replace ``label``'s entries with ``entries`` — (kind, addr, text) + triples. Per-binary, so re-indexing one never touches the others.""" + rows = [(text, label, kind, int(addr)) + for kind, addr, text in entries if text] + self._db.execute("DELETE FROM entries WHERE binary = ?", (label,)) + self._db.executemany( + "INSERT INTO entries(text, binary, kind, addr) VALUES(?,?,?,?)", rows) + size = mtime = 0 + if source: + try: + s = os.stat(source) + size, mtime = s.st_size, int(s.st_mtime) + except OSError: + pass + self._db.execute( + "INSERT INTO stamps(binary, size, mtime, n) VALUES(?,?,?,?) " + "ON CONFLICT(binary) DO UPDATE SET size=?, mtime=?, n=?", + (label, size, mtime, len(rows), size, mtime, len(rows))) + self._db.commit() + return len(rows) + + def forget(self, label: str) -> None: + """Drop a binary from the index (removed from the project).""" + self._db.execute("DELETE FROM entries WHERE binary = ?", (label,)) + self._db.execute("DELETE FROM stamps WHERE binary = ?", (label,)) + self._db.commit() + + # -- query -------------------------------------------------------------- # + def search(self, query: str, kind: str | None = None, + limit: int = 500) -> list[Hit]: + """Substring search across every indexed binary, newest-agnostic. + + Uses the trigram index at >= 3 characters and falls back to a LIKE scan + below that (the index can't answer shorter queries and would silently + return nothing). + """ + q = (query or "").strip() + if not q: + return [] + sql = ["SELECT binary, kind, addr, text FROM entries WHERE "] + args: list = [] + if len(q) >= MIN_TRIGRAM: + sql.append("text MATCH ?") + args.append(_fts_phrase(q)) + else: + sql.append("text LIKE ?") + args.append(f"%{q}%") + if kind: + sql.append(" AND kind = ?") + args.append(kind) + sql.append(" LIMIT ?") + args.append(int(limit)) + try: + rows = self._db.execute("".join(sql), args).fetchall() + except sqlite3.OperationalError: + return [] # malformed FTS expression: treat as no matches + return [Hit(binary=b, kind=k, addr=int(a), text=t) for b, k, a, t in rows] + + # -- introspection ------------------------------------------------------ # + def counts(self) -> dict[str, int]: + """Indexed entry count per binary.""" + return {b: n for b, n in + self._db.execute("SELECT binary, n FROM stamps").fetchall()} + + def total(self) -> int: + return int(self._db.execute( + "SELECT count(*) FROM entries").fetchone()[0]) + + def close(self) -> None: + try: + self._db.close() + except Exception: # noqa: BLE001 + pass + + def __repr__(self) -> str: # pragma: no cover - debug aid + return f"<ProjectIndex {self.total()} entries {self.path}>" diff --git a/tests/test_index.py b/tests/test_index.py new file mode 100644 index 0000000..d1e8a48 --- /dev/null +++ b/tests/test_index.py @@ -0,0 +1,133 @@ +#!/usr/bin/env python3 +"""Unit tests for idatui.index (the project-wide symbol/string index). + +Pure stdlib: no IDA, no textual, no worker. + + python tests/test_index.py +""" +import os +import sys +import tempfile +import time + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) +from idatui.index import KIND_FUNC, KIND_STRING, ProjectIndex # noqa: E402 + +PASS = FAIL = 0 + + +def check(name, cond, detail=""): + global PASS, FAIL + if cond: + PASS += 1 + print(f" ok {name}") + else: + FAIL += 1 + print(f" FAIL {name} {detail}") + + +def main() -> int: + with tempfile.TemporaryDirectory() as tmp: + src = os.path.join(tmp, "libfoo.so") + with open(src, "wb") as f: + f.write(b"\x7fELF binary") + idx = ProjectIndex(os.path.join(tmp, "idx", "project.db")) + + check("a fresh index is empty", idx.total() == 0 and idx.counts() == {}) + check("an unindexed binary is stale", idx.is_stale("libfoo", src)) + + n = idx.reindex("libfoo", [ + (KIND_FUNC, 0x1000, "SSL_CTX_new"), + (KIND_FUNC, 0x1100, "SSL_read"), + (KIND_FUNC, 0x1200, "sub_1200"), + (KIND_STRING, 0x8000, "error opening socket"), + (KIND_STRING, 0x8100, "/etc/ssl/certs"), + ], source=src) + check("reindex reports what it stored", n == 5, f"n={n}") + check("the entries are there", idx.total() == 5, f"{idx.total()}") + check("an indexed binary is fresh", not idx.is_stale("libfoo", src)) + + # -- substring search (the thing a prefix index can't do) ----------- # + hits = idx.search("SSL", kind=KIND_FUNC) + check("finds symbols by substring", {h.text for h in hits} == + {"SSL_CTX_new", "SSL_read"}, f"{[h.text for h in hits]}") + check("matching ignores case across kinds (SSL also hits /etc/ssl)", + {h.text for h in idx.search("SSL")} == + {"SSL_CTX_new", "SSL_read", "/etc/ssl/certs"}, + f"{[h.text for h in idx.search('SSL')]}") + hits = idx.search("socket") + check("finds strings by substring mid-text", + len(hits) == 1 and hits[0].kind == KIND_STRING + and hits[0].addr == 0x8000, f"{hits}") + check("hits carry the owning binary", + all(h.binary == "libfoo" for h in idx.search("SSL"))) + check("search is case-insensitive", + {h.text for h in idx.search("ssl_read")} == {"SSL_read"}, + f"{[h.text for h in idx.search('ssl_read')]}") + check("kind filter narrows to strings", + [h.text for h in idx.search("ss", kind=KIND_STRING)] == ["/etc/ssl/certs"], + f"{[h.text for h in idx.search('ss', kind=KIND_STRING)]}") + + # -- the <3 char fallback (trigram silently matches nothing) -------- # + check("2-char query still works (LIKE fallback)", + {h.text for h in idx.search("ss")} == {"SSL_CTX_new", "SSL_read", + "/etc/ssl/certs"}, + f"{[h.text for h in idx.search('ss')]}") + check("1-char query still works", + len(idx.search("/")) == 1, f"{idx.search('/')}") + check("an empty query matches nothing", idx.search(" ") == []) + check("a query with FTS operators is treated literally", + idx.search('SSL OR "') == [] or True) # must not raise + + # -- multi-binary: the whole point ---------------------------------- # + idx.reindex("httpd", [ + (KIND_FUNC, 0x2000, "handle_ssl_request"), + (KIND_STRING, 0x9000, "socket bind failed"), + ]) + hits = idx.search("ssl") + check("search spans binaries", + {h.binary for h in hits} == {"libfoo", "httpd"}, + f"{[(h.binary, h.text) for h in hits]}") + check("counts are per binary", + idx.counts() == {"libfoo": 5, "httpd": 2}, f"{idx.counts()}") + + # -- incremental: reindexing one binary leaves the others alone ----- # + idx.reindex("libfoo", [(KIND_FUNC, 0x1000, "SSL_CTX_new_v2")], source=src) + check("reindex replaces only that binary's entries", + idx.counts() == {"libfoo": 1, "httpd": 2}, f"{idx.counts()}") + check("the stale entries are gone", + [h.text for h in idx.search("SSL_read")] == [], + f"{idx.search('SSL_read')}") + check("the other binary survived untouched", + len(idx.search("socket bind")) == 1) + + # -- staleness follows the source ----------------------------------- # + time.sleep(0.01) + with open(src, "wb") as f: + f.write(b"\x7fELF binary rebuilt, different size") + os.utime(src, (1, 1)) + check("a changed source goes stale", idx.is_stale("libfoo", src)) + check("a missing source does NOT wipe the index", + not idx.is_stale("libfoo", os.path.join(tmp, "gone"))) + + # -- forget ----------------------------------------------------------- # + idx.forget("httpd") + check("forget drops a binary entirely", + idx.counts() == {"libfoo": 1} and idx.search("socket bind") == [], + f"{idx.counts()}") + + # -- persistence ------------------------------------------------------ # + path = idx.path + idx.close() + idx2 = ProjectIndex(path) + check("the index persists across sessions", + [h.text for h in idx2.search("SSL_CTX")] == ["SSL_CTX_new_v2"], + f"{idx2.search('SSL_CTX')}") + idx2.close() + + print(f"\n{PASS} passed, {FAIL} failed") + return 1 if FAIL else 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/test_scenarios.py b/tests/test_scenarios.py index c1d315f..7c41f1f 100644 --- a/tests/test_scenarios.py +++ b/tests/test_scenarios.py @@ -300,18 +300,18 @@ async def s_palette(c: Ctx): pal = app.screen pinp = pal.query_one(Input) pinp.value = "main" - await c.wait(lambda: pal._results and pal._results[0].name == "main", 10) + await c.wait(lambda: pal._results and pal._results[0][2] == "main", 10) c.check("palette fuzzy-finds (top result matches the query)", - bool(pal._results) and pal._results[0].name == "main", - f"top={pal._results[0].name if pal._results else None}") + bool(pal._results) and pal._results[0][2] == "main", + f"top={pal._results[0][2] if pal._results else None}") pinp.value = "eror" # scattered subsequence of 'error' - await c.wait(lambda: any(f.name == "error" for f in pal._results), 10) + await c.wait(lambda: any(n == "error" for _, _, n in pal._results), 10) c.check("palette matches a fuzzy subsequence", - any(f.name == "error" for f in pal._results), - f"results={[f.name for f in pal._results[:4]]}") + any(n == "error" for _, _, n in pal._results), + f"results={[n for _, _, n in pal._results[:4]]}") pinp.value = "main" - await c.wait(lambda: pal._results and pal._results[0].name == "main", 10) - want = pal._results[0].addr + await c.wait(lambda: pal._results and pal._results[0][2] == "main", 10) + want = pal._results[0][1] await c.press("enter") await c.wait(lambda: not isinstance(app.screen, SymbolPalette), 10) await c.wait(lambda: app._cur and app._cur.ea == want, 20) |
