aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* README: cut the wall of text out of 'What's in it'blasty2 days1-67/+37
| | | | | | | | | | | | | | | | | | | | The feature coverage was right, the prose was not: eleven paragraphs that explained MECHANISM in an intro document. The keys table right above it already proved that scannable beats thorough here. Every feature is now name + key + at most two lines. Longest block went from ~70 words to 35; the section from 80 lines to 53. What got cut was rationale, not information, and none of it is lost -- each one is already the docstring of the module that implements it: why findings has to journal edits (journal.py), how search guesses text vs bytes (search.py), trail painting and unseen bytes as ?? (trace.py), why terminal graphics support is asked for rather than sniffed (kittygfx.py). The README now says what a feature IS and links onward. Also folded hex/strings/palettes/projects/splash into one 'Also' line (all already in the keys table), and gave the same treatment to the Run prose and the agent-skill paragraph.
* Ship the idatui agent skill in-repo at .agents/skills/blasty2 days2-1/+425
| | | | | | | | | | | | | | | | | | | | | | | | | Follows the Agent Skills standard (agentskills.io) rather than a contrib/ dir: harnesses discover .agents/skills/<name>/SKILL.md automatically from the repo root, and it is vendor-neutral (not .claude/ or .pi/ specific). Not a verbatim copy of the local one -- it was ~85% generic and the rest had to go before redistributing: * the backend is described as a normal pyproject dependency, with the editable-git-checkout setup demoted to a conditional note (that is one maintainer's layout, but the 'upstream pull silently swaps the backend' warning is worth keeping for anyone who does it); * personal agent tooling (bgrun/waitfor wrappers) replaced by the underlying lesson -- a backgrounded subshell holds the inherited stdout, so redirect all three fds -- which is true without those scripts; * maintainer commit authorship dropped; * absolute ~/dev paths made relative to the repo root; ~/ida-venv kept but marked as this project's assumption, matching the README. Also fixes the README's stale full-run count (788/~2m -> 1031/~50s), lists the docs that were missing from the index, and warns that a user-level skill of the same name shadows this one -- otherwise a personal copy silently wins and the in-repo copy rots.
* SPEED: replace the historical backend table with a real 0.3.1 vs 0.3.2 A/Bblasty2 days2-8/+155
| | | | | | | | | | | | | | | | | | | | | | | | The worker-vs-Code-Mode table was measured before 0.3.2 and with both workarounds active, so it answered a question nobody asks any more. Replaced with three configurations measured on the same box, rolling both checkouts back and forward: A old client WITH workarounds on 0.3.1 -- what shipped B current client on 0.3.1 -- what the workarounds were for C current client on 0.3.2 -- now Headline: the real-world gain is ~1.4x geomean, NOT the 6.9x the empty round trip advertises, and the doc says so in those words -- because the tempting number to quote is the wrong one. The A->C vs B->C gap is the actual story: stock 0.3.1 was 5.4x slower, so the workarounds had already recovered nearly everything and upstream mostly bought us the right to delete them. Also records the three cost classes (payload- / round-trip- / IDA-dominated) so the next person optimising here knows which lever moves which op, and the ~10% run-to-run spread so a sub-1.2x 'regression' doesn't start a hunt. Old worker table kept below, labelled historical. experiments/bench_ops.py is the harness, with the copy-to-/tmp-before-checkout trick documented in it.
* Docs: re-check ALL nine upstream findings against 0.3.2blasty2 days2-23/+66
| | | | | | | | | | | | | | | | | | | | | Items 1-2 were marked fixed; 3-9 were left 'not re-verified', which was a gap rather than a finding. Checked all seven. Item 3 is also FIXED, and by a lot: a same-box A/B (roll the editable checkout back to 4195f21, measure, roll forward) puts execute_python('result = 1') at 2.055ms on 0.3.1 and 0.294ms on 0.3.2 -- 7.0x, now about the cost of a bare HTTP GET. The 0.3.1 column reproduces the originally documented 2.025ms almost exactly, which is what makes the 0.3.2 column believable. So the execute_sync marshalling that was ~93% of the floor is gone, and the 'never make a call per row' rule is much weaker. Items 4-9 are open by construction: client/registry/resolver/server/ database/worker .py are byte-identical between 0.3.1 and 0.3.2, and all six items live in those files. Recorded that as the cheap re-check recipe for the next upstream pull. SPEED.md taught the obsolete 2ms floor as a design rule; corrected, with a warning not to go rewrite working batched code to be chatty.
* Docs: correct the stale --fast check counts (302/257 -> 380)blasty2 days3-4/+4
|
* Docs: mark the two upstream perf findings fixed in 0.3.2blasty2 days2-9/+41
| | | | | | | | CODEMODE_UPSTREAM items 1 (timeout_trace) and 2 (to_jsonable) both landed upstream; note it at the top, on each item and in the priority table, and flag that items 3-9 are not re-verified against 0.3.2. SPEED.md described the settrace workaround and its IDATUI_CODEMODE_TRACE knob as current -- both are gone, and its backend table predates 0.3.2.
* Drop the settrace workaround: ida-codemode 0.3.2 deleted the hookblasty2 days2-41/+125
| | | | | | | | | | | | | | | | | | | | _script() ran every snippet with sys.settrace(None) to detach Code Mode's per-line trace hook, which returned itself and so enabled line tracing in every frame it saw -- 52x on ida_bytes.get_flags, and the whole residual gap against the old private worker. 0.3.2 removes that hook entirely (zero settrace refs); the deadline is now a C-level thread interrupt, runtime._interrupt_thread. Re-measured on a 200-row listing page of main in targets/bash: the workaround buys 0.99x. So it goes, and the caveat goes with it -- a pure-Python loop inside a snippet is once again bounded by its deadline. _PACK_EPILOGUE measures 0.97x for the same reason (dumps_json got a C fast path) but is kept for encoder determinism, with its stale 114x claim corrected. experiments/bench_pack_trace.py is the harness for both numbers. Full gate green: 1031 passed, 0 failed.
* Graph: never hand triskel a block its root cannot reachidatui2 days3-58/+125
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | "EMPTY BL" in the status bar is triskel's own bracket-list assertion from its SESE pass, and it turned out to be the mild version of the problem. Triskel's graph root is whichever node was created FIRST, and every analysis walks out from it. Anything unreachable from that node is undefined behaviour. We were: - creating nodes in id order, so the root was the lowest-numbered block rather than the entry, and - splitting only WEAKLY connected components, which says nothing about reachability. A 7-block CFG whose entry has no successors -- IDA hands those out for thunks and for dead code it could not resolve -- SEGFAULTS the interpreter. That is unsurvivable: it takes the session down and there is no exception to fall back from. Now the entry is created first, orphan blocks are attached to it with phantom edges that steer placement but are never drawn (one edge usually adopts a whole orphan subgraph, attached at a node no other orphan reaches), and reachability is asserted in python BEFORE crossing into C++. This replaces the component splitting entirely: one layout instead of N stacked side by side, and triskel gets to place the orphans. The reproducer is now a test (t_unreachable_entry). Remaining fallbacks on the ls corpus are 8/1200 layouts, all the upstream box-overlap defect, all but one on 300-500 block functions.
* Graph: fix the triskel fallbacks, and say why when it still falls backidatui2 days6-32/+182
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "engine triskel -> native+triskel-failed" was unreadable and, worse, gave no reason: the cause went to a logger a TUI user never sees. Dumped all 400 functions of bin/ls (echo's 128 were not enough) and swept them at three zoom levels: 6 of 1200 layouts fell back, all of them my own _verify tripping over a detour that could not be placed. - the detour jumped to the nearest side of the FIRST box in the way, which in a dense layout is usually inside the next box along. It now collects every box the run passes and picks the nearest genuinely free line. - it skipped the first and last segments because they carry the port and the arrowhead. But that is exactly where the failures were: triskel is happy to park a block directly above its successor and drive the final approach straight through it. Those segments may now move ALONG their own box's border, which is free almost every time. - repairs are swept to a fixed point: moving one segment stretches its neighbours, which can push those into a box. 0/1200 fallbacks after that. Then the bigger corpus turned up a second, genuinely upstream defect: superimposing SESE regions can leave two blocks a couple of columns into each other (2 of ls's 400 functions, in float space, before rounding). Cosmetic in a PNG; here the boxes are made of text, so one block's disassembly overwrites another's. _verify now checks it and falls back, which is the right trade. Reporting, so this is never mute again: - stats["engine_error"] carries the reason, the status line shows it, and the label is "native (triskel failed)". - the corpus test asserts fallbacks are rare AND explained, rather than asserting they never happen. Also lowered AUTO_TRISKEL_MAX_BLOCKS 250 -> 180. Layout runs on every zoom keypress and triskel knees hard past ~175 blocks (174: 66ms, 233: 489ms, 329: 555ms). The old cap allowed a 489ms stall. The corpus timing check now measures only sizes `auto` can actually reach, plus a 5s ceiling so nothing blows up quadratically when forced.
* Graph: name the interpreter when pytriskel is missingidatui2 days2-2/+17
| | | | | | | | | ./ida-tui runs $IDATUI_PYTHON (default ~/ida-venv), which is not the repo .venv the tests use -- so the graph view can report pytriskel as missing while tests/test_graph.py is exercising both engines. "not installed" on its own sends you to check the wrong python; the status now names sys.executable, and docs/GRAPH_VIEW.md says to install into both.
* graph_triskel: name the detour target, drop the unused swapidatui2 days1-8/+8
|
* Graph: a second layout engine, triskel's SESE decompositionuser2 days11-57/+1054
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `e` in graph mode cycles auto -> native -> triskel, and `auto` prefers triskel where it is installed and the function is at most 250 blocks. Why: our layered engine draws wide-and-short pictures with a lot of crossings on anything branchy. Triskel splits the CFG into Single-Entry Single-Exit regions first and lays each out on its own, which on the 128-function corpus means fewer crossings on 12 functions, equal on 9, worse on 3 -- and the wins are the hairballs (sub_5CA0 41 -> 6, sub_2C90 32 -> 7, sub_2C00 12 -> 0). It also routes loop edges around the side of the graph the way IDA does, which was a known gap here. It is not free: ~2x slower at 87 blocks, 10x at 424, hence the cap. The library needed a fork (~/dev/triskel, branch idatui) before it could be used from Python at all -- its get_waypoints() threw on every published version, an empty graph segfaulted the interpreter, and its spacing constants were pixels baked in at compile time. Making those settable is what makes this integration cheap: we hand it CELLS, so its output is integral and two edge lanes can never round onto the same row. The feared quantisation problem measured out backwards -- cells claimed by more than one edge: native 131, triskel 35. Not trusted with degenerate input, all handled before the call: self-loops and disconnected components make it throw, and one corpus edge comes back routed through a block, which we detour and re-verify. A triskel failure is never fatal; it falls back to native. Two things the second engine flushed out of the existing code: - the canvas was sized from boxes alone, which is exact only because native's dummy nodes reserve the space. Triskel routes outside that bounding box and the edges were being clipped. - arrowhead placement read e.back, conflating "this is a loop edge" (style) with "this polyline runs against control flow" (geometry). Now Edge.flipped, which is also a latent fix for residual-cycle edges whose succ/pred were being reported backwards. tests/test_graph.py runs its whole suite once per available engine (943 checks); new graph_engine scenario covers the live toggle.
* splash: scale the logo to the pane instead of dropping itblasty4 days3-10/+105
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reported as "the splash logo stopped rendering". It had not stopped: the splash asks for the artwork's NATURAL size and shows nothing when that does not fit, and the artwork needs 31 rows plus 10 of box chrome. A pane in a split zellij window is 31 rows — one row short of the 41 it wanted — so the logo silently disappeared. Traced with $IDATUI_KITTY_LOG in the real session: compose: supported=True app.size=Size(width=159, height=31) cells=60x23 fits=False The terminal scales an image into whatever cell box it is placed in (`c=`/`r=` on the placement), so there was never a reason for all-or-nothing. `logo_cells(max_rows)` now fits the art to the room left after the box's furniture, and the same number reserves the cells and sizes the placement, so a resize needs no relayout. In that same 31-row pane it now draws 55x21 instead of nothing. Two things fixed on the way: * The chrome constant was one row optimistic (`rows + 9` where the box measures 10: border 2, padding 2, art margin 1, title 1, note 1+1, help 1+1). At exactly the old threshold the help line was clipped off the bottom. * `_fits` conflated "is the terminal big enough" with "is the artwork the right size", which is what made the image path inherit the block art's all-or-nothing behaviour. The block art genuinely cannot scale (it is half-block cells, 26 rows) and still falls back to the text splash; the image no longer does. `splash_scaling` pins it at 31, 30 and 44 rows: the logo is drawn, it is scaled to the room, the box is never clipped, and a big pane still gets the natural size. 905 passed, 0 failed.
* Centre modals with a rule about modals, not a list of themblasty4 days3-10/+63
| | | | | | | | | | | | | | | | | | | | SearchPalette opened pinned to the top of the screen: the CSS named the screens that centre (`SymbolPalette, StringsPalette, ProjectPalette, …`) and a new dialog is not on a list nobody remembers to edit. The comment sitting above that rule — "every #pal-box palette centres, not just the symbol one" — was the *first* time this happened. `ModalScreen { align: center middle; }` matches subclasses, so every dialog inherits it and the next one is centred for free; the eight per-screen rules that only repeated it are gone. Textual's own Ctrl+P CommandPalette is a ModalScreen too and wants its stock top alignment, so it opts out in one visible line rather than by omission. The `modal_centering` scenario checks both halves: that centring is expressed as a rule, and that it actually reaches a dialog's laid-out region (above/below and left/right within a cell). 894 passed, 0 failed.
* Ctrl+F: search the whole database, by text or by bytesblasty4 days10-4/+728
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `/` only ever searched the lines of the view you were in. This adds the search you actually need on a binary: over the entire database, either through the rendered disassembly or through the image. * **text** matches the line as displayed, whitespace-normalised, so `call cs:` finds `call cs:getenv_ptr` (IDA's column padding is not something anyone types). Smartcase; `regex` available over RPC. * **bytes** is IDA's own `find_bytes`, so the pattern language people already know works unchanged: hex pairs, `?` wildcards for a whole byte or one nibble (`48 8? ?? 24`), quoted literals (`"Hello", 0`). Commas, no separators (`488B05C3`) and ragged spacing all normalise. **Which mode you meant is guessed, and the guess is biased on purpose.** `dead`, `add`, `cafe` and `ff` are valid hex AND ordinary things to search for, so a bare hex-looking word stays TEXT; nobody types `48 8b ?? c3` meaning prose. `hex:`/`text:` prefixes and F2 override it. The subtle case is a *typo* in a byte pattern. `48 zz c3` first fell through to a text search and reported "no match" — indistinguishable from "those bytes are not in this binary", which is the most misleading answer a search can give. Now any query whose tokens are all byte-sized is treated as bytes, and a bad token is refused BY NAME. IDA does the same thing quietly (find_bytes answers a malformed pattern with zero hits and no error), so the validation lives in Program.search, not just in the UI. Enter searches, then Enter opens the highlighted hit; the title says which it will do, because a database-wide scan is far too slow to run on every keystroke like the other palettes. Navigation goes to the item head — a byte match can start mid-instruction — and the status names the exact address. Also: the `find` RPC verb and `drive find`, which is the one an agent wants (`drive find '48 8b ?? c3'`). idatui/search.py holds the classification and is pure, so the whole question of "what did they mean" is tested offline: tests/test_search.py, 35 checks, 0.1s. Pilot scenario db_search covers the UI end to end. Full suite: 890 passed, 0 failed, 51.3s.
* Export findings as markdown (Ctrl+E), and the journal that makes it trueblasty4 days12-6/+1077
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The output of an RE session is what you worked out, and it was locked in a .i64 that only IDA can read. Ctrl+E (or `drive export`, or the `export` RPC verb) writes it out: your comments grouped by function with the line each annotates, the names and prototypes you set, the types you declared. **The hard part was provenance, and it needed a mechanism, not a filter.** A database does not record WHO wrote a comment or a name. IDA's analyzer sets `; switch 73 cases` and `; s1` with the same `set_cmt` a person uses, and the ELF loader sets `elf_gnu_hash_nbuckets` and `File class: 64-bit` the same way. Four probes, all negative: the FF_COMM flag is identical, `get_cmt` returns them all, `generate_disasm_line` tags every one of them COLOR_REGCMT (not COLOR_AUTOCMT), and they survive with auto-comments switched off. A first cut filtered by shape and produced a report whose first screen was ELF header trivia and `; jumptable ... case 99`. So idatui journals its own edits (idatui/journal.py) into a netnode in the database: it rides along in the .i64, it is still there next session, and the report is then exactly what was done here -- 2 findings out of a database carrying 693 other annotations. Recorded at the choke points in edit_ctl (rename, name-address, comment, retype) and in the struct editor; flushed on save, on export and on quit, so no edit pays a round trip. Without a journal (a database worked on in the IDA GUI, or predating this) the report falls back to filtering by shape -- dummy names, imports, loader segments, the analyzer's stereotyped switch/jumptable strings -- and says so in the document rather than claiming authorship it cannot prove. idatui/findings.py splits gather (needs IDA) from render (does not), so the formatting, grouping, sorting, escaping and the empty cases are tested offline: tests/test_findings.py, 32 checks, no worker, 0.1s. The pilot scenario covers the round trip that matters -- edit through the UI, export, find it in the file, and reload the journal from the .i64. Full suite: 842 passed, 0 failed, 51.2s.
* SPEED.md: the 85ms keypress, and what settle() still cannot seeblasty4 days1-21/+77
|
* tests: gate on real signals, not sleeps (117s -> 49s)blasty4 days6-60/+261
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The suite spent its time in two kinds of guess. **Flat pauses.** ~140 `pause(d)` calls were 20.4s of the pilot's 62s, and `test_trace_ui` was 13.5s of `pilot.pause(1.0)` out of 19.6s. `Ctx.pause` is now `settle` (`d` is the upper bound, not the cost) and the other suites' sleeps became gates on the thing the check is about. `Ctx.sleep` stays for what a timer really drives. **Textual's keypress path.** `Pilot.press` calls `wait_for_idle` twice per key, which sleeps in 20ms granules until process time stops advancing -- 84ms per keypress here, 23s of the pilot's 43s. `_fixtures.fast_keys()` replaces it with the gate the suites already use: send the keys, then settle. Deleting the heuristic *without* that broke nine checks, so it was doing a job, badly. Four checks turned out to be riding on those sleeps: they read geometry or a repaint (`si.region`, `gv._minimap_rect()`, glyphs off `gv.render_line`, a repaint trace), and a settled app has not necessarily been laid out or painted. They now wait for the frame. The debounced function filter (`set_timer(0.08)`) likewise waits for its effect. Also fixed two waits on signals that never arrive: the comment wait in `rename` carried a `dec.loaded_ea == app._cur.ea` conjunct that cost 9s of timeout and then let the check pass vacuously, and `listing_view` -- the one entry under "Known-flaky" -- waited on `lst.total`, which is true before a single row exists. `--profile` reports, per scenario, seconds settling / waiting / pressing, and names any wait that expired with its line number. It is how the above was found and how the next 20s should be. Verified: 4 full `tests/run.py` runs, 800 passed each, 49.0-49.2s (was 117.4s); 4 consecutive pilot runs, 313 passed each, 21.2s (was 63.7s).
* Struct editor: '/' fuzzy-filters the struct listblasty4 days3-10/+221
|
* Modal chrome: thin round borders, titles in the borderblasty4 days3-56/+223
|
* Syntax-highlight the struct editor's C definitionsblasty4 days4-14/+143
|
* demo: tighten the pacing (43s of pauses -> 19s)blasty4 days1-38/+38
| | | | | | | | | | Manual navigation is snappy; the demo was not, because I authored the pauses for someone reading every status line. Halved every beat with a 0.25s floor so the scenes stay distinguishable, and dropped the prompt typing from 45ms to 18ms per character -- still legibly typing, no longer something you wait on. --speed still scales it either way: 0.5 for a fast cut, 2.0 to slow it back down for a narrated take.
* demo: do not re-navigate to main when the app already landed thereblasty4 days1-3/+11
| | | | | | | The app auto-lands on main once the function index is in, so scene_open typed a goto to a place it was already standing. Every keystroke a driver injects costs a round trip, so that was pure dead time at the very start of the tour -- the worst place to spend it, since it is the part a recording opens on.
* demo: --here, render the TUI in the terminal that ran the demoblasty4 days1-13/+89
| | | | | | | | | | | | | | | | | | | | | | | | | --spawn needs tmux or zellij and puts the app in a second pane, which is wrong when the capture is one terminal. --here launches the TUI through the ./ida-tui launcher with stdio inherited, so it draws on the terminal you ran the demo from, then drives it over the socket exactly as before and quits it at the end. Going through the launcher rather than re-deriving the interpreter keeps the $IDATUI_PYTHON rules in one place. Two things this mode has to get right: - Narration would be drawn over the app, so it is collected and replayed after the alternate screen is gone. The operator still sees the full scene list, just at the end instead of during. - The socket is waited for with a deadline that also watches the child, so a TUI that dies before it ever listens fails in a second instead of hanging until the timeout. Teardown reaps the process (wait, then terminate, then kill) and removes the scratch copy on every path. --here and --spawn are rejected together. Verified: argument handling, the mutual exclusion, and that the pure suites are unaffected. The end-to-end --here run is NOT verified here -- it needs a real terminal, which this environment does not have.
* tools/demo.py: a scripted feature tour, and the RPC bug it foundblasty4 days3-1/+339
| | | | | | | | | | | | | | | | | | | | | | | | | | Ten scenes over the RPC layer: goto, listing scroll, pseudocode, the literal format ring, follow/back, the graph (zoom, minimap, edge walking), split view, xrefs, a rename and a comment, the strings/symbols/structs/hex browsers, and search. It drives the semantic verbs, so prompts are typed character by character and a recording shows the app being used rather than poked. --spawn opens its own pane on a COPY of the target (binary plus .i64, so the analysis is not re-paid) and tears it down afterwards; --sock drives a session you set up yourself, for control over pane size and zoom. Edits are reverted at the end, including a re-navigate so the last frame does not still show the demo's rename. --speed scales every pause, --only runs a subset, --list prints the scenes. Writing it found a real bug in the RPC hex verb: it waited for `app.is_hex` to become true, but backslash TOGGLES the hex view, so the call that leaves hex could never satisfy its predicate and always timed out after 20s. Any driver could open the hex view and never close it. It now waits for the mode to flip, the way toggle_view already did. Verified by running every scene against a live database and asserting the UI actually moved -- landed on main, decomp reachable, graph reports 280 blocks, split toggles off, no modal left open, the rename reaches the database and the revert removes it. 15/15. Full suite still 788 passed, 0 failed.
* Install ida-codemode from PyPI, not a sibling cloneblasty4 days3-27/+20
| | | | | | | | | | | | | | | | | | | | | | | | The README told people to "git clone ... ../ida-codemode" and pyproject carried a matching [tool.uv.sources] path override. That is not a convention, it is Duncan's development layout, inherited from the port patch and never questioned: it writes into the parent of the user's checkout, breaks if the directory is named anything else, and pins everyone to whatever happens to be in that working tree. ida-codemode is published on PyPI (0.3.1, the same version the clone was sitting at), so the dependency is now an ordinary versioned requirement and "uv sync" is the whole install. Verified end to end: uv lock resolves, uv sync --frozen installs from the registry, and ida_codemode.client.DatabaseHandle imports from the resulting venv. The path override is still the right answer when hacking on both at once, so that stays in the README as one "uv add --editable" line rather than as the default everyone pays for. Also fixed the plugin command it now documents: upstream's own README passes --prerelease=allow, and without it uvx refuses to resolve, because ida-codemode depends on ida-domain>=0.5.1.dev2. Checked that --install-plugin is a real flag and that the command runs, instead of copying it over on faith.
* docs: drop two stale files, correct the two the README links, cut sharp edgesblasty4 days5-542/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | README: "Known sharp edges" removed. docs/GRAPH_VIEW.md pointed the graph's one backend call at server/patch_server.py, a file the port deleted. It is idatui/codemode_client.py now. This matters because GRAPH_VIEW.md is one of only two docs the README links -- a wrong path in a linked doc is worse than no doc. docs/RPC.md was missing five verbs rpc.py serves: ping/methods/quit (now a Session table) and trace/binaries/switch (with the semantic verbs). The trace verb drives a whole feature the README advertises, so its absence was the real gap. Every verb in rpc.py's METHODS table is now documented, checked mechanically -- the remainder of what looked missing turned out to be values of the "move" verb's dir parameter, not verbs. Deleted: - docs/CODEMODE_PORT.md. Written mid-port, and its Result section claims the port covered "the complete current TUI feature set". It did not: the graph view was dead, the operand-format keys were dead, pseudocode comments never appeared, and bulk rename was broken. Keeping a doc that states the opposite of what we found is worse than having none, and docs/CODEMODE_UPSTREAM.md carries the accurate account of the API limits. - docs/TUI_DRIVING_BLUEPRINT.md. A design sketch for generalizing the drive-a-TUI-over-a-socket pattern to other programs, with gdb as the main example -- which now exists for real in ~/dev/gdb-driver. Superseded by the thing it proposed. Nothing linked either file. Both remain in git history.
* README: pure markdown, and a logo sized for a READMEblasty4 days2-8/+4
| | | | | | | | | | | | | The <img> tag existed only to set a width, and the <div align=center> only to centre things -- neither of which markdown can express. Sizing the image file itself removes the reason for both: logo-trans.png was a 1024x1024 1.7MB master being scaled down in the browser on every page load, and is now 400x400 and 219KB. Nothing else uses it (the splash reads logo.png and logo.ans), and the original is in git history if it is ever wanted back. So the README is now plain markdown end to end: ![](), no divs, and the footer is a link instead of a centred <sub>. Left-aligned rather than centred, which is what markdown does.
* README: drop the Layout section, trim Docs to the two worth openingblasty4 days1-23/+2
| | | | | | | | | Layout was internal orientation, not something a reader of the README needs; the source is right there. Docs listed six files, four of which are working notes from problems already solved (paging findings, Textual pitfalls, the port's own coverage doc, the upstream findings) -- interesting once, noise in a front page. What is left is the two a user actually opens, and both are still linked inline from the feature that needs them.
* README: rewrite, and fix the install it documentedblasty4 days3-200/+480
| | | | | | | | | | | | | | | | | | | | | | | | The old one had drifted into a wall of prose that described the project to someone who already knew it. Rewritten around what a reader actually needs: what this is, what it needs, how to run it, the keys, and what is in it -- with the logo up top and the sharp edges at the bottom instead of a wall of disclaimers at the top. Corrected while checking every claim rather than trusting it: - pyproject depended on 'ida-codemode-mcp' from '../ida-codemode-mcp'. The package is 'ida-codemode' and the repo is '../ida-codemode', so 'uv sync' -- the documented install -- could not have worked for anyone. Fixed, and uv.lock regenerated (it now resolves, 39 packages). - test counts were 257/733, they are 302/788 - the keys table is generated from the real BINDINGS: c/d/a/p/u, o/O/B, s for split, ctrl+t for structs, " for strings, all verified against app.py - dropped the ida-pro-mcp/worker-era architecture section, which described files that no longer exist, and replaced it with the current layout - links docs/CODEMODE_UPSTREAM.md and .fastfeedback/SPEED.md, which did not exist when the README was last touched Every path and doc link in it is checked to exist.
* Stop tracking the upstream port patchblasty4 days2-5848/+3
| | | | | | It was swept into the tree by a broad 'git add -A' during the port. The merge itself plus docs/CODEMODE_PORT.md are the record; a 251KB patch of superseded content is not. Still on disk, now ignored.
* Merge the IDA Code Mode portuser4 days40-2812/+9215
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replaces the private idalib worker (idatui/worker.py + worker_client.py, with server/patch_server.py injecting tools into ida-pro-mcp) with an ordinary client of ida_codemode.client.DatabaseHandle. A database open by an IDA GUI is reused; otherwise Code Mode starts or shares a managed idalib worker. The TUI no longer owns an IDA process, and closing it releases only its lease. Based on Duncan Ogilvie's port, rebased onto ~150 commits of local work it predated. The rebase itself was mechanical; landing it was not. Nine defects had to be fixed before the feature set was whole again, none of which the patch's own tests could catch: - DatabaseHandle.open() takes image_base, not loading_address: every connect() would have raised TypeError on the first call - five operations our tree had grown were simply missing (flowchart, so the graph view was dead; op_format/pc_nums/pc_num_format, so 'o'/'O' were; survey_binary) - set_comments wrote only the disassembly comment, so comments never appeared in pseudocode - xref_query returned rows in raw IDA order, and 'follow the call' silently followed the fall-through instead - rename accepted one edit per category, so bulk symbol import was dead - decompile ran decomp_map's full per-column ctree sweep to fill in a per-line address anchor - heads shipped without operand extents or the digest protocol - the package became unimportable without ida_codemode installed, which killed the offline test suites Verified against the pre-codemode tag rather than against assumptions: the full suite is 788 passed / 0 failed, and the pilot's 301 checks match the old backend exactly. Performance is within 2x on the listing hot path and faster on decompile, disasm and connect, after fixing two runtime costs that are documented for upstream in docs/CODEMODE_UPSTREAM.md. Test runtime came down from ~9m20s to 115s along the way -- not by removing checks, but by removing four kinds of waiting-on-a-guess that were also hiding real failures.
| * docs: upstream findings for the ida-codemode maintainersblasty4 days1-0/+269
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Nine items found while porting a latency-sensitive client, each with the measurement behind it and a suggested fix, ordered by what it cost us. The two that matter: timeout_trace returns itself, which turns on line tracing in every frame (52x on ida_bytes.get_flags, 10x on a listing page), and to_jsonable walking large results (114x what json.dumps of the same data costs). Together they were the difference between 35x slower than the worker we replaced and within 2x of it. Both live in the runtime, so every client pays them until they are fixed centrally -- we only found them because we had an in-process implementation to compare against. Also records the things we had to reach into internals for (waiting for a lease to release), the failure modes that are silent or fatal rather than typed (loader switches on an existing IDB, an IDB replaced under a live lease), and the one feature we could not port at all (discard-and-quit, since a managed worker saves when its final lease closes).
| * SPEED.md: where the last 2ms goes, and why batching is not worth buildingblasty4 days1-1/+30
| | | | | | | | | | | | | | | | | | | | Measured so nobody re-derives it: HTTP is 0.165ms of the 2.0ms floor and execute_sync marshalling onto IDA's main thread is the other 1.86ms, inside IDA's own serve() loop. Call volume is already minimal (8 calls to scroll 2000 rows, 4 for a 1060-block graph), so a batch endpoint would save single-digit ms on flows costing hundreds. The largest single operation in the app -- decompiling a 17KB function -- is 10806ms here and 10723ms on the worker: pure Hex-Rays, 0.8% apart, no transport in it.
| * SPEED.md: record the post-fix backend numbers and the 2ms round-trip floorblasty4 days1-17/+26
| |
| * codemode: close the performance gap with the old worker (heads 35x -> 2.2x)blasty4 days1-5/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Two changes, both about work that was never ours to do, found by profiling the A/B benchmark rather than guessing. 1. Serialise inside the database process. Code Mode runs to_jsonable() over whatever a snippet returns, walking the entire structure to make it JSON-safe. Our answers are already JSON-safe and they are large: a 200-row listing page is ~10k small objects, and walking them cost 66ms of the page's 92ms -- 114x what json.dumps of the very same data costs (0.58ms). Snippets now return one pre-serialised string, so that walk is O(1) and the client parses a payload it was going to parse anyway. heads(200): 92ms -> 24.7ms. 2. Detach the runtime's trace hook while our snippet runs. ida_codemode.runtime wraps every execute_python in sys.settrace(timeout_trace) to enforce deadlines, and timeout_trace RETURNS ITSELF -- which switches on LINE tracing in every frame it sees. Every line of every function we call pays a Python-level callback. Measured here: ida_bytes.get_flags 0.106us untraced 5.49us traced 52x (plain idalib, no Code Mode: 0.119us -- i.e. untraced == native) heads(200 rows) 2.0ms untraced 20.2ms traced 10x That one hook was the entire residual gap against the old unix-socket worker. The snippet now detaches it and restores it in a finally. What that gives up, stated plainly: the deadline is no longer enforced for a pure-Python loop inside our snippet. The runtime's other cancellation path -- a threading.Timer calling ida_kernwin.set_cancelled() -- does not go through the trace and still fires, so a long IDA operation remains interruptible, and every operation here is bounded by its own count/limit argument. Set IDATUI_CODEMODE_TRACE=1 to keep the stock behaviour. Against the worker backend, same box, targets/echo (worker -> codemode): heads_200 2.65ms -> 5.85ms 2.2x (was 35x) heads_500 6.21ms -> 10.67ms 1.7x heads expect-hit 2.16ms -> 4.61ms 2.1x disasm_200 9.03ms -> 5.25ms 0.6x faster decompile_cold 162.62ms -> 30.66ms 0.2x faster decompile_warm 30.02ms -> 25.39ms 0.8x faster decomp_map 45.82ms -> 47.85ms 1.0x parity pc_nums 19.90ms -> 22.56ms 1.1x parity rename_func 254.08ms -> 255.54ms 1.0x parity connect 550.0ms -> 410.0ms 0.7x faster What is left is the transport floor: an empty execute_python round trip is 2.0ms, so trivial calls (data_type 0.07ms -> 2.63ms, force_recompile, a single xref query) look like 40x while being 2.5ms of wall clock. Reducing those needs fewer calls, not faster ones -- the digest/expect path already does that for the listing, which is where call volume actually is. Full suite: 788 passed, 0 failed, 115.3s (was 146.3s; the pilot alone went 80.9s -> 62.2s).
| * Record the test-speed findings in .fastfeedback/SPEED.md (force-added)blasty4 days1-0/+203
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The suite went from ~9m20s (with thumb_ui crashing and rawimage_rpc failing) to 146s, 788 passed / 0 failed, without deleting a single check. Every second of that came from four mistakes, all of which will be made again by the next person who does not know about them, so they are written down next to the commands: 1. waiting on a signal that can no longer happen (model-identity gates, after edits were changed to keep the listing's walk) -- 30s in blob_ui, 4x60s in thumb_ui, each followed by a check that then passed vacuously; 2. waiting on a signal that is set too early (app._t is assigned before the navigation it starts) -- a race master's millisecond backend won and Code Mode's loses; 3. regenerating a fixture with os.urandom into a temp dir, so the pristine database cache could never apply; 4. deleting a database and reopening the same path, which races the lease of the worker that still owns it. It also records the per-suite timings, which suite owns which behaviour (batch rename lives ONLY in test_rawimage_rpc -- four full pilot runs missed that bug), and two harness traps: `bgrun run` blocks the agent's tool call unless launched with setsid and all three fds redirected, and `pkill -f` matches the agent's own command line and kills the shell issuing it. Force-added over the .gitignore entry: this is the kind of thing that is worth more to the next reader than it costs in repo noise. Drop the file and the ignore rule together if that judgement is wrong.
| * tests: trace_ui followed a signal that was not one (and it really did fail)blasty4 days1-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "the code view follows the trace" failed on the port (None vs 0x2ae4) and passed on master, but it was not a trace regression: the test pressed "]", waited for `app._t == 1`, and then read the listing cursor. `app._t` is assigned the moment the key is handled -- the navigation it starts runs in a worker -- so the wait was satisfied before the view had moved, and the check read a cursor that had no address yet. Master won that race because its backend answers in single-digit milliseconds. The Code Mode backend is slower, so the race became a reliable failure. The gate is now the condition the check is about (the cursor is on the trace's ip), for both the forward and the backward step. 39 passed, 0 failed -- the same tally as master.
| * tests: thumb_ui 5m13s (crashing) -> 8.5sblasty4 days1-37/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It was not slow because ARM analysis is slow. It was slow because it fought Code Mode's ownership model and then waited out timeouts for the result. The suite deleted <fixture>.i64 and REOPENED THE SAME PATH before each of its four phases. That was safe when the TUI owned a private worker that died with it. Under Code Mode the database is leased, and the previous phase's managed worker can still hold it through its lease grace -- so the delete raced a live owner, the next open never produced a listing, and the suite died on `lst.model.index_of_ea(0)` with model=None after burning minutes in waits whose results were never checked. Each phase now gets its own temp copy: separate paths cannot collide and nothing waits for anyone to let go. This is the same hazard docs/CODEMODE_PORT.md flags -- sweeping files that another client may own -- showing up in the test suite rather than in the app. Also replaced four `wait(lambda: lst.model is not <old>, ..., 60)` gates. An item edit now keeps the listing's walk instead of rebuilding it, so the model object is never replaced and each of those sat out its full 60 seconds while the suite still reported success. They now settle() on the signal the checks actually read: the status line announcing Thumb/ARM/64-bit, and the function appearing in the index. 20 passed, 0 failed (it never reached a tally before). Verified separately that the four ARM operations the port carried over do work against a live database: set_thumb, thumb_scan, define_code_run, define_func_run.
| * tests: blob_ui 39.8s -> 4.1s, and a fatal it was hidingblasty4 days4-29/+136
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Three separate wastes, all of the same family: waiting on a guess instead of a signal, and paying for work that never had to be repeated. 1. The 64KB blob was built with os.urandom into a fresh TemporaryDirectory on every run. New bytes at a new path means the pristine-database cache can never apply, so full auto-analysis of 64KB of AArch64-decoded noise was paid every single run. It is now built from a seeded PRNG at a stable path (tests/.synthetic/, gitignored) and staged through the existing cache. Determinism is also a correctness fix: whether 64KB of chance bytes contains something IDA reads as a function is luck, and this suite asserts "and really has no functions". 2. `wait(lambda: lst.model is not old, ..., 30)` after commenting. The perf work made an item edit KEEP the listing's walk and re-render in place, so the model object is never replaced and this waited out its full 30s timeout on every run -- and then "commenting leaves the view where it was" passed vacuously, because nothing had happened at all. A test that burns 30s to check nothing is worse than no test. 3. Two `pause(2.0)`/`pause(2.5)` after a carve, replaced with settle() on a real condition. The second one deliberately has NO predicate: that spot is random data, so the carve may legitimately produce nothing, and "the row became code" would never hold -- gating on it cost another 30s timeout. What that check is about is the VIEW not moving, so the gate is "the app finished reacting". Fixing (1) exposed a real bug in the client, fixed here too: reopening a database that already exists while passing loader switches is FATAL in IDA -- FATAL ERROR: Switch '-b400' can be used only when loading a new file which kills the worker before it can report anything. Loader switches describe an IMPORT and are recorded in the database they produce, so they are now sent only when there is an import to describe. This was never reachable from the old suite (a fresh random blob never had a database to reopen), but it is reachable by any user who opens a raw blob with --ida-args twice. 30 passed, 0 failed.
| * codemode: three defects the A/B benchmark found in the decompiler pathblasty4 days2-66/+198
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Benchmarking the port against master op-by-op (rather than only asking whether tests pass) turned up three real bugs, all in the most user-visible path: opening pseudocode. 1. decompile was doing decomp_map's job. It called the full per-column line map purely to fill in each line's /*0xEA*/ anchor. The tool ida-tui was written against takes ONE get_line_item at column 0 per line; the port took one per COLUMN, i.e. thousands of get_line_item+dstr() calls per function instead of one per line. Every pseudocode open cost the same as opening the split view. Carried the real implementation over: 1888ms -> 53ms. 2. decomp_map used the pre-optimisation line map. Ours memoises obj_id -> ea for the whole function (commit 853d90c: dstr() was 79% of the tool, and consecutive columns report the same ctree item), the port's did not. 1925ms -> 287ms. 3. _idatui_compact imported ida_pro_mcp on every call. Under Code Mode that package is not installed in the database process, so the import failed every time -- and a FAILED import is never cached, so each one re-searched the whole of sys.path: 422 failed imports per pc_nums call, which was most of its runtime. 1428ms -> 257ms. The same bug was a correctness bug hiding behind the perf bug: the fallback path collapsed whitespace INSIDE string literals, where the real function preserves it. Pseudocode columns are served in those coordinates, so on any line containing a string with two spaces, every literal's mark and every reformat would have been placed on the wrong column. It never fired on master because ida_pro_mcp is installed there. Now calls the byte-identical module-level shim directly, with the deviation from the extracted original documented in place. Narrow verification: decomp/split_view/opfmt/follow/comment/structs scenarios, 72 passed, 0 failed. Full gate running separately.
| * codemode: rename takes a LIST of edits per category, not just oneblasty4 days2-26/+120
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Found by tests/test_rawimage_rpc.py, which the earlier runs had not covered: every rename_many check failed with {"ok": 0, "failed": 2, "errors": [{"addr": null, "error": "list indices must be integers or slices, not str"}]} The port's rename read each category as a single edit (edit["addr"]), but the batch shape is {func: [{addr,name}, ...], data: [...], local/stack: [...]} -- a list per category, with a single dict accepted as shorthand. Indexing the list with "addr" raised, and because the whole category was one try block the error came back attached to addr=null, naming nothing. That is the entire point of the rename_many RPC verb: a firmware image arrives with hundreds of names from a loader map or an emulator's symbols.json, and applying them one at a time costs a navigation plus two prompt round trips each. Only the single-rename UI path worked. Now mirrors the real tool: one row per EDIT (addr/old/name plus a per-row error), a summary counting edits rather than categories, conflict detection before the write, and dry_run/allow_overwrite/stop_on_error. Renaming a function refreshes Hex-Rays' ctext, whose cache is per function and persisted in the .i64 -- without it the pseudocode keeps calling the old name forever while every other readback reports the new one. Clearing a label with an empty new name is kept as a real request (the scenarios revert with it) rather than being rejected as a missing argument. tests/test_rawimage_rpc.py: 14 passed/7 failed -> 21 passed, 0 failed.
| * codemode: carry over the listing + operand-format tools, and stop reshipping ↵blasty4 days2-175/+1442
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | them This closes the five operations the port was missing and restores the listing's own tooling instead of a re-implementation of it. idatui/remote_tools.py is the port's IDAPython island: `heads` (the continuous listing) and `op_format`/`pc_nums`/`pc_num_format` (`o`/`O`), extracted verbatim from the BODY that server/patch_server.py used to inject. They are real, diffable source shipped to the database process as text, not string literals, because this is the most performance-tuned and behaviour-sensitive code in the project. Why carry `heads` over rather than keep the port's version: the port's rewrite emitted no per-operand extents ("ops"), so no keypress could show which literal it would reformat (opfmt_highlight had no two-operand row to find); it had no digest/`expect` support, so every page was re-sent after any edit; and its span walk was the per-character loop ours had already been rewritten out of. It also dropped struct-member expansion sizing and the func banner/label rows' exact shapes. The library is installed ONCE per database process (sys.modules, keyed by a hash of the source) and then called by name. Code Mode's execute_python builds a fresh namespace per call, so a library exec'd inline is rebuilt every time and its module-level caches thrown away -- the per-line render lru_cache in particular, which the perf work sized to 65536 entries. Installing it once took `heads` count=200 from 181ms to 92ms; the cache reports 211 hits on a second call where it previously reported none. (Extraction footgun recorded: ast FunctionDef.lineno points at `def`, not at the decorators, so a naive slice silently drops @lru_cache.) Also ported: flowchart, survey_binary, and the xref contract. Live pilot suite on targets/echo: 301 passed, 0 failed -- identical to master. Known, quantified, and NOT fixed here: Code Mode's transport is much slower than the unix-socket worker for the listing's paging. heads count=200 is 2.6ms on master vs 92ms here, count=500 is 6.3ms vs 214ms. Roughly half of that is to_jsonable + HTTP framing per call and is inherent to the architecture; the empty round trip alone is 2ms. The digest/`expect` path (unchanged pages) is the main mitigation and is restored.
| * codemode: port the xref tools' real contract, order includedblasty4 days1-32/+111
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The pseudocode follow's address fallback broke: following a call landed on the NEXT LINE instead of the callee (decomp_nav's stale-name check, cur=0x20dd want=0x2060). The port's xref_query returned rows in raw IDA order, and at a call site IDA yields the ordinary-flow xref (fl_F, the next instruction) before the call xref (fl_CN), so 'first code xref' picked the fall-through. The tool ida-tui was written against sorts rows by the far-end address and dedups by default; sorted, 0x2060 precedes 0x210e and the follow is correct. That ordering is load-bearing, so it is now part of the port rather than an accident of the old implementation. Also fixed: the port attached 'fn' to ref.from_ea for both directions, where a from-xref must describe its TARGET (the xref dialog shows the wrong function otherwise), and the envelope was missing direction/addr/total/next_offset/resolved_addr. xref_types (ours, the kind badges in the xref dialog) is ported verbatim and deliberately stays UNsorted -- that dialog lists xrefs in IDA's own order. decomp_nav, follow_xrefs, xref_labels, decomp_follow_self: 15 passed, 0 failed.
| * codemode: restore the graph view and pseudocode commentsblasty4 days1-5/+97
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Verified live now: ida-codemode 0.3.1 spawns a managed idalib worker on this box, so the pilot suite runs against the port. flowchart: the port simply does not have the operation, so domain.get_flowchart returned None and every graph key reported 'no control-flow graph for this function'. Ported ours onto ida_gdl (ida-domain exposes no basic-block or edge-kind surface). Blocks stay address RANGES, never text -- that is what lets graph boxes reuse the listing's own rows. Graph suite: 0 -> 50 passed. set_comments: the port set only the disassembly comment via db.comments.set_at(), so a comment never appeared in the pseudocode. A Hex-Rays comment is anchored to a ctree location and an anchor the ctree does not own is discarded as an orphan, so the itp slot must be searched until one sticks, and the entry ea is a function comment instead. Ported that logic back. survey_binary: added as the (caught) fallback domain.py expects behind file_regions, so the fallback path is real rather than always empty.
| * codemode: fix DatabaseHandle.open kwarg, and check kwargs against the real ↵blasty4 days2-2/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | signature ida-codemode is now cloned at ../ida-codemode (0.3.1) and installed into ~/ida-venv, so the adapter can be checked against the library instead of against assumptions. First thing it found: connect() passed loading_address=, which DatabaseHandle.open() does not have. The real parameter is image_base, and it already wants the natural 16-byte-aligned address we compute, so this is a rename. Every connect would have died with TypeError on the first call. The port's own contract test could not catch it: its fake handle takes **kwargs, so any keyword at all looks accepted. The test now also validates the keywords we send against inspect.signature(DatabaseHandle.open) when the library is importable, and skips that one check when it is not. Offline suite: 302 passed with the library installed, 302 without it.
| * Rebase MISTER EXO's ida-codemode port onto the current treeblasty4 days34-3992/+7999
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Mechanical part of the port: the 27-file patch was cut against a base ~148 commits behind us, so it did not apply. Resolved 11 conflicts (all of them diff drift, not semantic clashes) and the three file deletions: - app.py: the patch re-inserted _do_rename/_do_name_addr/_seek_split etc. as "theirs" because our tree moved them to edit_ctl.py/trace_ctl.py. Kept ours and applied the real intent (WorkerClient->CodeModeClient, .call->.invoke, _open_worker_client->_open_database_client) at their current homes. - domain.py: kept Head as a NamedTuple -- the patch reverted it to a frozen dataclass, which the perf work measured at 2.9us vs 1.9us per row on a quarter-million-row walk. Dropped _fetch_output (no download_url under Code Mode) and its now-dead urllib/json imports. - pane.py: the patch's deletion swallowed our zellij support along with the worker-reaping block it meant to remove. Kept zellij, removed the reaping. - test_scenarios.py: the idb_save->save_database teardown hunk belongs to tests/_fixtures.py in our tree; applied it there and kept our pc_num_format scenario that the drift landed on. Three defects in the patch itself, fixed here: - It made "import idatui" hard-require ida_codemode, so every offline suite died at import -- including the pure ones (graph/index/trace) that are the house rule for "tests/run.py --fast". The import is now deferred and gated on the binding, which is also what lets the port's own contract tests inject a fake DatabaseHandle. - project.stage() inlined an ida_codemode.registry import and treated "library not installed" as "someone owns this database", which broke IDA-free project staging. Ownership lookup moved to codemode_client.database_owner(). - tests/test_codemode_client.py had no NEEDS_IDA marker, which tests/run.py rejects outright. Offline suite: 301 passed, 0 failed. Against master's 344 the whole delta is accounted for: -40 worker_client (module deleted), -18 launch sweep checks (behaviour deliberately removed) +3 guarding that it stays removed, +2 pool (GUI-save semantics), +13 new codemode_client contract tests. NOT yet done, and the port is not functional without it: the adapter is missing five operations our tree grew since the patch's base (flowchart, op_format, pc_nums, pc_num_format, survey_binary) and its "heads" predates back-walking and digest/expect.
* Stop tracking 157MB of core dumps, and ignore thempre-codemodeuser5 days1-0/+4
| | | | | | Two core dumps were swept into 7e4b593 by an 'add -A' commit. idalib segfaults readily under differential probing (running two revisions of a tool against one cfunc drops two SWIG item objects on the same ctree), so this will recur.
* Park two proven-but-unresolvable F5-path optimisations, and record how to ↵user5 days5-0/+386
| | | | | | | | | | | | | | | | | | spot a counter-drift outlier pc_nums allocated three ctree_item_t SWIG objects per candidate column -- the third instance of the same fault already fixed in decomp_map and found in decompile_function_safe. 1247 -> 614ms warm over 18991 lines of bash, identical literal counts, 0 mismatches over echo's 128 functions. Both it and the fast decompile_function_safe are parked rather than applied: together they are worth ~350ms against a run-to-run spread of ~500ms on this box (means 25045 with, 25140 without over seven runs), so the benchmark cannot resolve them. Neither adds complexity -- both remove allocations -- so they are kept on disk with their measurements for a per-operation-latency goal. Also records the 27283ms outlier: the work counters moved with it, which is how an outlier is told from a regression.
* decomp_map: memoise obj_id -> ea for the whole function instead of only ↵user5 days3-14/+80
| | | | | | comparing against the previous column. dstr() was 79% of the tool (24us a call) and items interleave, so foo(a, b) flips call->arg->call and re-formatted an item already seen: 106594 calls for 15417 lines of bash. Also corrects run #30's claim that the duplicate ida_hexrays.decompile is what costs -- a warm decompile is 0.01ms. Result: {"status":"keep","total_ms":24513.8,"lg_boot_ms":693.2,"lg_decomp_ms":2347.3,"lg_graph_ms":931.2,"lg_hex_ms":460.3,"lg_index_ms":69.1,"lg_listing_cold_ms":434.2,"lg_listing_warm_ms":462.4,"lg_nav_ms":6639.6,"lg_palette_ms":4.7,"lg_rename_ms":699.9,"lg_render_ms":219.7,"lg_search_ms":3441.1,"lg_split_ms":2218,"pure_graph_ms":216.5,"sm_boot_ms":436.9,"sm_decomp_ms":1264.9,"sm_graph_ms":758.2,"sm_hex_ms":437.9,"sm_index_ms":2.3,"sm_listing_cold_ms":256.8,"sm_listing_warm_ms":256,"sm_nav_ms":309.5,"sm_palette_ms":0.3,"sm_rename_ms":379.1,"sm_render_ms":249.5,"sm_search_ms":59.2,"sm_split_ms":1266,"fails":0}
generated by cgit v1.3.1-sl0p (git 2.54.0) at 2026-08-11 21:05:20 +0000 — sl0p fork