| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Last commit let you SAY how to load a blob. This one notices when you should
have. Interactive IDA pops a dialog when no loader matches; we silently loaded
as x86 at 0 and analysed to nothing, so the flag only helped people who already
knew they needed it — which is exactly the people who don't need help.
formats.sniff() recognises the formats IDA definitely handles (ELF, PE, Mach-O,
dex, wasm, ar, COFF, Intel HEX, S-records). Anything else gets
LoadOptionsScreen: a filterable processor list with human labels, a load-address
field, Enter to accept, Esc to load it the way IDA would have anyway.
Deliberate asymmetry: the sniff only claims formats it is sure about. A false
"unknown" costs one dismissible dialog; a false "known" is the silent wrong
answer this exists to kill. Esc is always an escape hatch.
The list offers 20 processors, not IDA's 73 — most of the rest are museum
pieces, and a name typed into the filter that matches nothing is taken literally
so nothing is actually unreachable. Endianness is spelled out (arm vs armb)
because getting it backwards is the most common route to zero functions.
Asked only when nobody has answered yet: not with --processor, not in project
mode (entries carry their own), and not when a database exists — the .i64
already records how the image was loaded.
Textual trap worth recording: the screen stored the file size in self._size,
which is Widget's own backing field for outer_size. Assigning an int to it
crashes layout with "'int' object has no attribute 'region'" from deep inside
_set_dirty, nowhere near the cause. Same family as the _render collision.
The paragraph conversion now lives in exactly one place (formats.load_args);
BinaryRef and launch both call it.
Verified on a real AArch64 blob: dialog appears, filtering to "arm" leaves two
entries, base 0x8000000 accepted -> "-parm -b800000" -> 35 functions at
0x8002440. An ELF never asks, and neither does a blob that already has a .i64.
tests: new tests/test_formats.py (21) and a load_options scenario asserting the
dialog stays out of the way for a recognised binary. 199/0 scenarios, 39/0
project, 30/0 project UI, 36/0 index, 27/0 pool, 21/0 formats.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A raw firmware dump has no format to detect, so IDA fell back to x86 at address
0. It doesn't fail — it opens, analyses, and finds nothing. An AArch64 image
loaded this way gave 0 functions; told the truth it gives 35.
ida-tui fw.bin --processor arm --base 0x8000000
and per binary in a project, which is what a multi-image firmware actually
needs:
{"path": "app.bin", "processor": "arm", "base": "0x8000000"}
idapro.open_database() already accepted IDA command-line switches; nothing was
passing any. Plumbed BinaryRef -> WorkerPool -> WorkerClient -> worker argv, plus
a load_args for the single-binary path that has no project ref.
base is written the way people say it (0x8000000, int or string, any base).
IDA's -b is in PARAGRAPHS — -b1000 loads at 0x10000 — so BinaryRef.load_args
converts, and a base that isn't 16-byte aligned is refused rather than silently
landing 16x off. ida_args passes anything else through.
Two bugs found by testing the whole path rather than the happy one:
* Project.load() whitelisted path/label when normalising entries, so the load
options were dropped the first time a project was reopened — set a processor,
come back tomorrow, it's gone.
* Re-passing the switches to an EXISTING database makes IDA refuse the open
(rc != 0, no functions). The .i64 already records how the image was loaded, so
the worker skips them once a database exists. My first guard checked
splitext(path) + ".i64" and never fired, because IDA names it "<file>.i64" —
keeping the extension. It checks both spellings now.
Verified on a real AArch64 blob: fresh load 35 functions based at 0x8002440,
reopen 35 again, and the CLI rejects an unaligned or non-numeric --base.
tests: +6 project (options recorded, paragraph conversion, file round-trip,
add() takes them, an ELF passes nothing, hex-string base). 39/0 project, 195/0
scenarios, 30/0 project UI, 36/0 index, 27/0 pool.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
xrefs_to only ever sees the current database, so an exported function looks
unused from the inside even when the rest of the project calls it. The import
side of the phase-3 linkage index already knew better; nothing surfaced it.
_foreign_importers appends those callers to the xrefs dialog. From libc's
strrchr, with echo in the project:
0000C318 import [echo] strrchr
Read from the on-disk index, so a caller appears whether or not its worker is
resident. Choosing one carries a (binary, addr) payload instead of a bare
address; _on_xref_chosen routes that through _switch_then_goto — the same path a
project search hit takes — so it records a hop and Esc comes back.
Only fires for a symbol this binary actually EXPORTS. A local name that happens
to collide with some other binary's import is not a caller of ours, and without
that check every common name (main, read, error) would sprout fictional callers.
Names are compared after link_name(), so ELF versioning doesn't hide the match.
Verified on a real echo+libc project: the dialog lists echo's call site, and
selecting it switches to echo, lands on 0xc318 and leaves hops=['libc.so.6'].
tests: +3 project UI — a symbol we don't export gets no cross-binary callers,
the (binary, addr) payload jumps to the other binary, and it records the hop.
195/0 scenarios, 30/0 project UI.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Three things, all following from phase 3 making cross-binary jumps ordinary.
**A cross-binary jump was a one-way door.** Nav history is per-binary, so
arriving in another binary — a project search hit, or now following an import
into the library that implements it — landed you in an empty history with nothing
to take you back. _switch_then_goto records the binary it came FROM, and
action_back falls through to that hop once local history is spent: Esc walks back
through the function you were in, then the binary you were in. Manual Ctrl+O
switching records nothing, because that isn't navigation.
**Pre-warm follows the linkage graph, not list order.** _prewarm_provider warms
the binary providing the most of this one's imports — where a follow is most
likely to go, so its startup is paid before you ask for it. "Next in the list"
would have been arbitrary; phase 3 gave us something better to ask.
pool.prewarm() refuses rather than making room. Evicting a binary the user
visited to speculatively load one they haven't is a straight downgrade, and it
throws away that binary's caches as well; at a tight budget pre-warm just does
nothing. The cost of a worker that doesn't exist yet can only be estimated, so it
uses the largest resident one (same program, different database) — and if that
estimate proves wrong, the speculative worker is the one evicted, never a chosen
one.
**Driving a project.** pane spawn --project FILE [--open BIN]; `binaries` lists
the inventory (active / resident / indexed / where Esc returns to) and `switch
{binary,addr?}` makes another active — with an address it takes the search-hit
path, so it records a hop. state gains `binary` and `hops`, which it should have
had the moment project mode existed.
Verified on real sessions: drive binaries/switch against an echo+cat project
pane; Esc crossing back from a switch; and prewarm on echo+libc picking libc
(provider of echo's imports) and warming it after an evict.
tests: +5 pool (prewarm warms, no-ops when resident, refuses at budget, evicts
nothing when refusing, ignores unknown labels) and +4 project UI (jump records
the hop, Esc crosses back, hop consumed). Confirmed the Esc-back checks fail with
the branch removed. 195/0 scenarios, 27/0 project UI, 36/0 index, 27/0 pool,
33/0 project.
Left open: project-level persistence across sessions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Following a call to strcmp reached the PLT/extern entry and stopped there —
Hex-Rays has nothing to decompile, because the code lives in a library this
binary only references. With the other binary open in the same project we
already had everything needed to cross that gap; we just weren't indexing it.
Index each binary's imports and exports (KIND_IMPORT / KIND_EXPORT) alongside its
functions and strings. On a follow, _import_stub asks whether the target address
is one of this binary's import stubs; if so _cross_binary_impl asks the index who
exports that name, and we switch there instead of landing on the thunk.
Verified end to end on a real echo + libc project: Enter on `strrchr(a1, 47)` in
echo's pseudocode switches to libc.so.6 and lands on strrchr at 0xaf960.
Three things it turns on:
* ELF symbol versioning. The importer sees strrchr@@GLIBC_2.2.5 while the
provider may export any of three spellings, so raw names resolve almost
nothing. domain.link_name() cuts at the first '@'; Linkage.raw keeps what IDA
reported, which is what the listing shows.
* Exact match, not substring — ProjectIndex.exact(), so `read` doesn't bind to
pread/read_line/thread_start. It also answers below the 3-char trigram floor,
and plenty of real exports are that short.
* Resolution reads the on-disk index, so a provider resolves while its worker is
evicted. That's what the index was for.
When nothing in the project provides the symbol _follow_import declines and the
normal navigation runs: landing on the stub is still the honest answer, and a
single-binary session is unchanged. The PLT-stub PRESENTATION item stays open —
an unprovided import should say "imported, provider not in project" rather than
show a decompiler error.
server/patch_server.py gains list_linkage (idautils.Entries + enum_import_names);
a worker without it degrades to no linkage rather than failing.
tests: index join +8 (exact vs substring, short names, exclude-self, reverse
join, kind isolation, forget unresolves) and link_name +4. 36/0 index, 195/0
scenarios, 23/0 project UI, 33/0 project, 22/0 pool.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Phase 2: strings has the F2 toggle now, so drop the "still needs it" note.
Record the 60-row project cap and the reason for it, and the ranking rule —
including why the (binary, addr) tiebreak is load-bearing (a name shared by two
binaries ties on every other field and Python then compares hit objects,
raising TypeError; that shipped as a crash and was fixed).
* Phase 3: spell out precisely how it relates to the PLT/import-stub backlog
item. An export index turns "follow strcmp -> extrn strcmp:near, dead end" into
a real jump into libc WHEN that library is in the project. It doesn't retire
the item: a single-binary session, or a provider outside the project, still
lands on a stub and still wants it recognised and presented as an import rather
than a decompiler error.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
idatui/index.py — one on-disk index (<sidecar>/idx/project.db) over every binary
in a project, so search works for binaries whose worker isn't running.
Indexing choice, measured rather than guessed:
* SQLite FTS5 with the TRIGRAM tokenizer — stdlib, no dependency (nothing else
was installed and nothing is needed), and unlike a prefix index it matches
arbitrary substrings, which is what symbol names and string bodies need.
* 300k-entry corpus: 1.9 ms per query vs 11.8 ms for a Python scan and 28.9 ms
for plain LIKE; 0.2 ms per incremental insert.
* Size was the stated worry and turned out not to bite: bash contributes 5.9k
entries / 0.15MB of text, libcrypto.so.3 30.7k / 0.52MB. At ~5.7x the text a
20-binary project is ~12-23MB — against .i64 files already in the sidecar
(libcrypto's alone is 72MB), roughly 1% of what the project already costs. The
reason to be on disk is residency, not size.
* Trigram can't answer queries under 3 chars and returns nothing rather than
erroring, so search() falls back to LIKE — otherwise incremental typing would
look broken until the third keystroke.
Wiring: after a binary's functions load, its symbols + strings are folded into
the index (skipped when the source's size/mtime is unchanged). Ctrl+N gains a
scope toggle on F2 — not ctrl+a, which the focused Input binds to "home" so it
never reaches the palette. Project scope narrows via the index then ranks with
the existing _fuzzy, keeping the same feel; hits are prefixed with their binary,
and choosing one elsewhere switches binary and jumps to it.
Also fixes another instance of the Textual-markup trap: the palette titles ate
"[project]" as a style tag (same class of bug as the status bar), so the pal
titles are markup=False now.
tests/test_index.py: 24 stdlib checks — substring/case-insensitive matching, kind
filter, the <3 char fallback, multi-binary search, per-binary incremental
reindex, staleness, forget, persistence. Suite 191/0.
Strings (") still needs the same scope toggle; the index already carries them.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Wires the project model + worker pool into the app. Project mode is ADDITIVE —
without --project the app is exactly the single-binary tool it was, which is what
keeps the 167-check pilot meaningful.
* IdaTui(project=...) builds a WorkerPool and opens the project's first binary;
_open_worker_client asks the pool instead of spawning directly.
* BinaryState snapshots what a switch leaves behind (program, func_index, nav,
cur, view prefs, filter). Switching reuses the _after_reconnect shape: swap
client+program, rebuild the index, reopen the entry. A still-resident binary
restores instantly (Program + index are in memory); an evicted one gets a fresh
worker but keeps its nav history, which is just addresses.
* ProjectPalette (Ctrl+O, + a "Switch binary…" palette command): the project's
binaries with resident/analysed/pinned/active state and memory, filterable.
* launch.py --project FILE, creating the project when binaries are also given;
stages everything up front so the source tree is never written to.
Two bugs found while testing:
* _did_auto_land is app-wide, but landing is per-binary: after the first binary
landed, a cold switch never landed at all AND left the switch overlay up
forever. Reset it per switch.
* PRE-EXISTING: the status Static had Textual markup enabled, so a single-word
bracket marker parses as a style tag and is silently eaten — [listing] and
[pseudocode] have never actually rendered (only [split · listing] survived,
because the · makes it an invalid tag). Status is plain text with brackets and
symbol names, so markup=False.
tests/test_project_ui.py: end-to-end pilot on two real binaries (18 checks) —
boot, switcher contents, switch, per-binary index, both workers resident,
switch back with state intact, return-to-where-you-were, and the promise that
the source tree stays pristine while every artifact lands in the sidecar.
Full single-binary suite unchanged at 167/2 (the standing flakes).
|
|
|
First slice of multi-binary projects (docs/PROJECTS.md): the on-disk model, with
no runtime wiring yet.
idatui/project.py (stdlib-only, like domain/worker):
* Project.load/create/save — an explicit JSON project file listing binaries;
paths resolve relative to it, labels default to the basename and are
disambiguated on collision (they name files).
* A sidecar dir beside the project file (<stem>.idatui.d/) holds bin/ (staged
binaries), their .i64 + scratch, and idx/ for phase 2. IDA opens the STAGED
file, so nothing lands in the source tree — today targets/ carries ~244MB of
IDA litter around ~13MB of binaries, much of it stale wedge files.
* stage() copies rather than hardlinks. A hardlink is free but makes source and
staged one inode, so an in-place rebuild (cp over the path truncates instead of
replacing) would silently swap the bytes under an analysed DB with nothing to
detect it. The unit test caught exactly that. A copy also leaves the sidecar
self-contained once the sources are gone.
* Re-staging a changed source drops its now-stale DB; sweep_scratch() clears the
unpacked working files a hard-killed worker leaves behind (never the .i64).
tests/test_project.py: 27 checks, pure stdlib (no IDA/textual/worker), <1s.
docs/PROJECTS.md: the full design — the one-worker-per-DB constraint with
measured costs (bash worker = 126MB RSS/117MB PSS; libcrypto's DB is 72MB, so
residency is budgeted by MEMORY, not a worker count), the switch between
"switching needs a live worker" and "searching doesn't (cached index)", and
phases 1-4.
|
